Searching and retrieving multimedia information

• US 6,243,676 B1
• Filed: 12/23/1998
• Issued: 06/05/2001
• Est. Priority Date: 12/23/1998
• Status: Expired due to Term

• Current Assignee: Unwired Planet LLC (JP Morgan Chase & Co)
• Original Assignee: Openwave Systems Incorporated (JP Morgan Chase & Co)
• Inventors: Witteman, Bradley James
• Primary Examiner(s): Smits, Talivaldis I.
• Assistant Examiner(s): Nolan, Daniel A.
• Application Number:
  US09/221,334
• Time in Patent Office: 895 Days
• Field of Search: 704/271, 704/272, 704/276-278, 704/251, 704/231, 704/201, 704/243, 704/260
• US Class Current: 704/243
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting

Abstract

A method retrieves a multi-media segment from a signal stream having an audio component and a closed caption component. This includes separating the audio component and the closed caption text component from the signal stream, generating an audio pattern representative of the start of the multi-media segment, locating the audio pattern in the audio component, and temporally aligning the text from the closed caption text component with the audio pattern in the audio component. Locating the audio pattern in the audio component includes retrieving text from the closed caption text component; and comparing the text against one or more keywords delimiting the multi-media segment. Once located, the multi-media segment may be played on-demand. In addition, an apparatus retrieves a multi-media segment from a signal stream, the signal stream having an audio component and a closed caption text component. The apparatus includes a decoder for separating the audio component and the closed caption text component from the signal stream, an audio synthesizer coupled to the decoder for generating an audio pattern representative of the start of the multi-media segment, a pattern recognizer coupled to the decoder and to the audio synthesizer for locating the audio pattern in the audio component, and an aligner coupled to the pattern recognizer and to the decoder for temporally aligning the text with the audio pattern in the audio component. The apparatus also plays the multi-media segment on-demand.

Claims

1. A method for retrieving a multi-media segment from a signal stream, the signal stream having an audio component and a closed caption text component, the method comprising:

separating the audio component and the closed caption text component from the signal stream;

generating an audio pattern representative of the start of the multi-media segment;

locating the audio pattern in the audio component; and

temporally aligning the close caption text component with the audio pattern in the audio component.

2. The method of claim 1, further comprising:

retrieving the text from the closed caption component; and

comparing the text against one or more keywords delimiting the multi-media segment.

3. The method of claim 2, wherein the keywords delimit the start and the end of the multi-media segment.

4. The method of claim 2, wherein the keywords are stored in a dictionary.

5. The method of claim 2, further comprising comparing the text from the closed caption text component against one or more key characters delimiting the multi-media segment.

6. The method of claim 5, wherein the key characters are stored in a dictionary.

7. The method of claim 1, further comprising:

generating a concluding audio pattern representative of the end of the multi-media segment; and

locating the concluding audio pattern in the audio component.

8. The method of claim 7, further comprising identifying the multi-media segment between the audio patterns.

9. The method of claim 1, wherein the multi-media segment and closed caption text component are indexed are indexed in a database.

10. The method of claim 1, wherein the multi-media segment is played on-demand.

11. An apparatus for retrieving a multi-media segment from a signal stream, the signal stream having an audio component and a closed caption text component, the apparatus comprising:

means for separating the audio component and the closed caption text component from the signal stream;

means for generating an audio pattern representative of the start of the multi-media segment;

means for locating the audio pattern in the audio component; and

means for temporally aligning the text from the closed caption text component with the audio pattern in the audio component.

12. The apparatus of claim 11, further comprising:

means for retrieving the text from the closed caption text component; and

means for comparing the text against one or more keywords delimiting the multi-media segment.

13. The apparatus of claim 12, wherein the keywords delimit the start and the end of the multi-media segment.

14. The apparatus of claim 12, wherein the keywords are stored in a dictionary.

15. The apparatus of claim 12, further comprising means for comparing the text from the closed caption text component against one or more key characters delimiting the multi-media segment.

16. The apparatus of claim 15, wherein the key characters are stored in a dictionary.

17. The apparatus of claim 11, further comprising:

means for generating a concluding audio pattern representative of the end of the multi-media segment; and

means for locating the concluding audio pattern in the audio component.

18. The apparatus of claim 17, further comprising means for identifying the multi-media segment between the audio patterns.

19. The apparatus of claim 11, wherein the multi-media segment and the text are indexed in a database.

20. The apparatus of claim 11, further comprising means for playing the multi-media segment on-demand.

21. An apparatus for retrieving a multi-media segment from a signal stream, the signal stream having an audio component and a closed caption text component, the apparatus comprising:

a decoder for separating the audio component and the closed caption text component from the signal stream;

an audio synthesizer coupled to the decoder for generating an audio pattern representative of the start of the multi-media segment;

a pattern recognizer coupled to the decoder and to the audio synthesizer for locating the audio pattern in the audio component; and

an aligner coupled to the pattern recognizer and to the decoder for temporally aligning the text with the audio pattern in the audio component.

22. The apparatus of claim 21, further comprising:

a text extractor for retrieving text from the closed caption text component; and

a comparator for comparing the text against one or more keywords delimiting the multi-media segment.

23. The apparatus of claim 22, wherein the keywords delimit the start and the end of the multi-media segment.

24. The apparatus of claim 22, further comprising a dictionary for storing the keywords.

25. The apparatus of claim 22, wherein the text from the closed caption text component is compared against one or more key characters delimiting the multi-media segment.

26. The apparatus of claim 25, further comprising a character dictionary for storing the key characters.

27. The apparatus of claim 21, wherein the synthesizer generates a concluding audio pattern representative of the end of the multi-media segment and the pattern recognizer locates the concluding audio pattern in the audio component.

28. The apparatus of claim 27, further comprising a database for storing the multi-media segment between the audio patterns and the text.

29. A method for retrieving a multi-media segment from a signal stream, the signal stream having an audio component and a closed caption text component, the method comprising:

generating audio patterns representative of the start and the end of the multi-media segment;

locating the audio patterns in the audio component; and

delimiting a portion of the audio component between the audio patterns as the multi-media segment.

30. The method of claim 29, further comprising:

temporally aligning the text from the closed caption text component with the start of the delimited audio component portion; and

storing the text from the closed caption text component and the delimited audio component portion in a database.