Video cataloger system with audio track extraction

• US 7,295,752 B1
• Filed: 02/05/2002
• Issued: 11/13/2007
• Est. Priority Date: 08/14/1997
• Status: Expired due to Fees

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: Virage, Inc. (HP Inc.)
• Inventors: Fuller, Charles, Horowitz, Bradley, Gupta, Amarnath, Gorkani, Mojgan Monika, Hampapur, Arun, Jain, Ramesh, Portuesi, Michael J., Shu, Chiao-fe, Humphrey, Richard D., Bach, Jeffrey
• Primary Examiner(s): Miller; John
• Assistant Examiner(s): ONUAKU, CHRISTOPHER O
• Application Number:
  US10/067,550
• Time in Patent Office: 2,107 Days
• Field of Search: 386/39, 386/46, 386/52, 386/54, 386/65, 386/68, 386/69, 386/83, 386 95- 98, 386101-106, 386124-126, 386/112, 386/111, 348/14.1, 348/423, 348/425.1, 348/467, 348460-462, 348/512, 348/515, 345/302, 345/327, 360/4, 360/72.1, 369/30.04, 369/30.18, 369/32, 369/275.3, 704/250, 704/253
• US Class Current: 386/285
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7834 using audio features
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G11B 27/107 of operating tapes
• CPC Class Codes: G11B 27/323 Time code signal, e.g. on a cue track as SMPTE- or EBU-time code
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/84 Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

One aspect of the invention is directed to a system and method for video cataloging. The video is cataloged according to predefined or user definable metadata. The metadata is used to index and then retrieve encoded video.

Claims

1. A method of extracting audio for indexing of video, comprising:

receiving video information having embedded audio information and associated time codes;

capturing the embedded audio information in the video information;

extracting a plurality of audio metadata tracks from the audio information, each audio metadata track having selected ones of the time codes indicative at least of start and stop times for the audio metadata track;

encoding the video information; and

accessing the encoded video information with the selected time codes of one of the audio metadata tracks.

2. The method of claim 1, wherein the video information is received from an analog source.

3. The method of claim 2, wherein the analog source is a videotape deck.

4. The method of claim 2, wherein the analog source is a live satellite feed.

5. The method of claim 1, wherein the video information is received from a digital source.

6. The method of claim 1, wherein the capturing includes digitizing with an audio digitization devices.

7. The method of claim 1, wherein the plurality of audio metadata tracks includes at least one of:

keywords, speech-to-text transcription, speaker identification and audio class.

8. The method of claim 1, wherein the time codes comprise SMPTE codes.

9. The method of claim 1, wherein the encoding comprises encoding with an MPEG format.

10. The method of claim 1, wherein the audio metadata tracks comprise different types of audio metadata tracks.

11. An audio engine for extracting metadata tracks, comprising:

an audio signal switch receiving an audio signal;

an audio classification component controlling the audio signal switch according to whether the audio signal is classified as speech; and

a plurality of audio metadata track extraction components in data communication with the output of the switch, wherein each audio metadata track extraction component provides an audio metadata track associated with speech.

12. The audio engine of claim 11, additionally comprising:

an audio capture component for capturing and digitizing an analog audio source; and

an audio signal normalization component for normalizing the digitized audio prior to processing.

13. The audio engine of claim 11, wherein the audio metadata tracks include at least one of:

keywords, speech-to-text transcription and speaker identification.

14. The audio engine of claim 11, wherein the audio classification component additionally classifies at least silence and music.

15. The audio engine of claim 11, wherein the audio metadata track extraction components receive data from a customizable dictionary.

16. The audio engine of claim 11, wherein the audio signal is received from a real-time source.

17. The audio engine of claim 11, wherein the audio signal is received from a digital source.

18. The audio engine of claim 11, wherein the audio signal is received from a digital camcorder.

19. The audio engine of claim 11, wherein each audio metadata track extraction component provides a different type of audio metadata track.

20. An audio engine for extracting metadata tracks, comprising:

an audio signal switch receiving an audio signal;

an audio classification component in data communication with and controlling the audio signal switch according to whether the audio signal is classified as speech; and

a plurality of audio metadata track extraction components in data communication with the output of the switch, wherein each audio metadata track extraction component provides an audio metadata track associated with speech.

21. The audio engine of claim 20, wherein the audio metadata tracks include at least speaker identification.

22. The audio engine of claim 20, wherein the audio classification component additionally classifies at least music.

23. The audio engine of claim 20, wherein the audio metadata track extraction components receive data from a customizable dictionary of data associated with the extracted metadata tracks.

24. The audio engine of claim 20, wherein the audio signal is received from a remote real-time source.

25. The audio engine of claim 20, wherein the audio signal is received from a remote digital source.

26. The audio engine of claim 20, wherein each audio metadata track extraction component provides a different type of audio metadata track.

27. A method of extracting audio for indexing of video, comprising:

receiving video information having embedded audio information and associated time codes;

capturing the embedded audio information in the video information;

extracting a plurality of audio metadata tracks from the audio information, each audio metadata track being associated with selected ones of the time codes indicative at least of start and stop times for the audio metadata track;

encoding the video information; and

accessing the encoded video information with the selected time codes of one of the audio metadata tracks.

28. The method of claim 27, wherein the video information is received from a remote digital source.

29. The method of claim 27, wherein the plurality of audio metadata tracks includes at least one of:

keywords, speech-to-text transcription, speaker identification and audio class.

30. The method of claim 27, wherein the audio metadata tracks comprise different types of audio metadata tracks.

31. An audio engine for extracting metadata tracks, comprising:

an audio signal switch receiving an audio signal;

an audio classification engine;

an audio class dictionary configured to provide dictionary data indicative of audio classes to the audio classification engine;

an audio class profiler in data communication with the audio classification engine, wherein the audio class profiler receives the audio signal, and wherein the audio class profiler is further in data communication with and controls the audio signal switch according to whether the audio signal is classified as speech; and

a plurality of audio metadata track extraction components in data communication with the output of the switch, wherein each audio metadata track extraction component provides an audio metadata track associated with speech.

32. The audio engine of claim 31, additionally comprising:

an audio capture component for capturing and digitizing an analog audio source; and

an audio signal normalization component for normalizing the digitized audio prior to processing.

33. The audio engine of claim 31, wherein the audio metadata tracks include at least one of:

keywords, speech-to-text transcription and speaker identification.

34. The audio engine of claim 31, wherein the audio class profiler additionally classifies at least silence and music.

35. The audio engine of claim 31, wherein the audio metadata track extraction components receive data from a customizable dictionary of data associated with the extracted metadata tracks.

36. The audio engine of claim 31, wherein the audio signal is received from a real-time source.

37. The audio engine of claim 31, wherein the audio signal is received from a digital source.

38. The audio engine of claim 31, wherein the audio signal is received from a digital camcorder.

39. The audio engine of claim 31, wherein the audio class dictionary is customizable.

40. The audio engine of claim 31, wherein each audio metadata track extraction component provides a different type of audio metadata track.