Multimedia visualization and integration environment

• US 20040175036A1
• Filed: 03/03/2004
• Published: 09/09/2004
• Est. Priority Date: 12/22/1997
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey
• Granted Patent: US 8,995,767 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06F 16/30 of unstructured textual data document management systems G06F16/93
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Embodiments of the present invention provide techniques for retrieving and displaying multimedia information. According to an embodiment of the present invention, a graphical user interface (GUI) is provided that displays multimedia information that may be stored in a multimedia document. According to the teachings of the present invention, the GUI enables a user to navigate through multimedia information stored in a multimedia document. The GUI provides both a focused and a contextual view of the contents of the multimedia document.

Claims

1. A computer-implemented method of displaying recorded information, the recorded information comprising audio information and video information, the method comprising:

receiving information identifying a word;

displaying a representation of the video information; and

displaying a representation of the audio information wherein one or more occurrences of the word in the representation of the audio information are highlighted.

2. The method of claim 1 wherein displaying the representation of the audio information comprises:

transcribing the audio information to generate a transcription;

displaying a representation of the transcription; and

highlighting each occurrence of the word in the displayed representation of the transcription.

3. The method of claim 1 wherein displaying the representation of the audio information comprises:

receiving closed-caption text corresponding to the audio information;

displaying a representation of the closed-caption text; and

highlighting one or more occurrences of the word in the displayed representation of the closed-caption text.

4. The method of claim 1 wherein displaying the representation of the video information comprises highlighting one or more portions of the displayed representation of the video information that are relevant to the word.

5. The method of claim 1 wherein displaying the representation of the video information comprises:

extracting a plurality of frames from the video information;

determining one or more frames from the plurality of frames that comprise the word; and

displaying the plurality of frames extracted from the video information,

wherein the one or more frames are highlighted.

6. The method of claim 5 wherein determining the one or more frames from the plurality of frames that comprise the word comprises:

for each frame in the plurality of frames;

using an optical character recognition technique to determine text contained in the frame; and

determining if the text determined from the frame comprises the word.

7. A computer-implemented method of displaying recorded information, the recorded information comprising audio information and video information, the method comprising:

receiving information identifying a topic;

displaying a representation of the video information; and

displaying a representation of the audio information, wherein one or more portions of the representation of the audio information relevant to the topic are highlighted.

8. The method of claim 7 wherein displaying the representation of the audio information comprises:

transcribing the audio information to generate a transcription;

displaying a representation of the transcription; and

highlighting one or more portions of the transcription that are relevant to the topic.

9. The method of claim 8 wherein highlighting the one or more portions of the transcription that are relevant to the concept comprises:

determining one or more words from the transcription that are relevant to the topic; and

highlighting the one or more words.

10. The method of claim 7 wherein displaying the representation of the audio information comprises:

receiving closed-caption text corresponding to the audio information;

displaying a representation of the closed-caption text; and

highlighting one or more words in the displayed representation of the closed-caption text that are relevant to the topic.

11. The method of claim 7 wherein displaying the representation of the video information comprises highlighting one or more portions of the displayed representation of the video information that are relevant to the topic.

12. The method of claim 7 wherein displaying the representation of the video information comprises:

extracting a plurality of frames from the video information;

determining one or more frames from the plurality of frames that are relevant to the topic; and

displaying the plurality of frames extracted from the video information,

wherein the one or more frames are highlighted.

13. The method of claim 12 wherein determining the one or more frames from the plurality of frames comprises:

for each frame in the plurality of frames;

using an optical character recognition technique to determine contents of the frame; and

determining if the contents of the frame are relevant to the topic.

14. A computer program product stored on a computer readable medium for displaying recorded information, the recorded information comprising audio information and video information, the computer program product comprising:

code for receiving information identifying a word;

code for displaying a representation of the video information; and

code for displaying a representation of the audio information wherein one or more occurrences of the word in the representation of the audio information are highlighted.

15. The computer program product of claim 14 wherein the code for displaying the representation of the audio information comprises:

code for transcribing the audio information to generate a transcription;

code for displaying a representation of the transcription; and

code for highlighting each occurrence of the word in the displayed representation of the transcription.

16. The computer program product of claim 14 wherein the code for displaying the representation of the audio information comprises:

code for receiving closed-caption text corresponding to the audio information;

code for displaying a representation of the closed-caption text; and

code for highlighting one or more occurrences of the word in the displayed representation of the closed-caption text.

17. The computer program product of claim 14 wherein the code for displaying the representation of the video information comprises code for highlighting one or more portions of the displayed representation of the video information that are relevant to the word.

18. The computer program product of claim 14 wherein the code for displaying the representation of the video information comprises:

code for extracting a plurality of frames from the video information;

code for determining one or more frames from the plurality of frames that comprise the word; and

code for displaying the plurality of frames extracted from the video information, wherein the one or more frames are highlighted.

19. The computer program product of claim 18 wherein the code for determining the one or more frames from the plurality of frames that comprise the word comprises:

for each frame in the plurality of frames;

code for using an optical character recognition technique to determine text contained in the frame; and

code for determining if the text determined from the frame comprises the word.

20. A computer program product stored on a computer readable medium for displaying recorded information, the recorded information comprising audio information and video information, the computer program product comprising:

code for receiving information identifying a topic;

code for displaying a representation of the video information; and

code for displaying a representation of the audio information, wherein one or more portions of the representation of the audio information relevant to the topic are highlighted.

21. The computer program product of claim 20 wherein the code for displaying the representation of the audio information comprises:

code for transcribing the audio information to generate a transcription;

code for displaying a representation of the transcription; and

code for highlighting one or more portions of the transcription that are relevant to the topic.

22. The computer program product of claim 21 wherein the code for highlighting the one or more portions of the transcription that are relevant to the concept comprises:

code for determining one or more words from the transcription that are relevant to the topic; and

code for highlighting the one or more words.

23. The computer program product of claim 20 wherein the code for displaying the representation of the audio information comprises:

code for receiving closed-caption text corresponding to the audio information;

code for displaying a representation of the closed-caption text; and

code for highlighting one or more words in the displayed representation of the closed-caption text that are relevant to the topic.

24. The computer program product of claim 20 wherein the code for displaying the representation of the video information comprises code for highlighting one or more portions of the displayed representation of the video information that are relevant to the topic.

25. The computer program product of claim 20 wherein the code for displaying the representation of the video information comprises:

code for extracting a plurality of frames from the video information;

code for determining one or more frames from the plurality of frames that are relevant to the topic; and

code for displaying the plurality of frames extracted from the video information, wherein the one or more frames are highlighted.

26. The computer program product of claim 25 wherein the code for determining the one or more frames from the plurality of frames comprises:

for each frame in the plurality of frames;

code for using an optical character recognition technique to determine contents of the frame; and

code for determining if the contents of the frame are relevant to the topic.

27. A system for displaying recorded information, the recorded information comprising audio information and video information, the system comprising:

a processor module; and

an output module coupled to the processing module,

wherein the processor module is configured to;

receive information identifying a word,

cause the output module to display a representation of the video information, and

cause the output module to display a representation of the audio information wherein one or more occurrences of the word in the representation of the audio information are highlighted.

28. The system of claim 27 wherein the processor module is configured to:

extract a plurality of frames from the video information;

determine one or more frames from the plurality of frames that comprise the word; and

cause the output module to display the plurality of frames extracted from the video information, wherein the one or more frames are highlighted.

29. A system for displaying recorded information, the recorded information comprising audio information and video information, the system comprising:

a processor module; and

an output module coupled to the processing module,

wherein the processor module is configured to;

receive information identifying a topic,

cause the output module to display a representation of the video information, and

cause the output module to display a representation of the audio information, wherein one or more portions of the representation of the audio information relevant to the topic are highlighted.

30. The system of claim 29 wherein the processor module is configured to:

extract a plurality of frames from the video information;

determine one or more frames from the plurality of frames that are relevant to the topic; and

cause the output module to display the plurality of frames extracted from the video information, wherein the one or more frames are highlighted.