SCALABLE SUMMARIES OF AUDIO OR VISUAL CONTENT

• US 20080300872A1
• Filed: 05/31/2007
• Published: 12/04/2008
• Est. Priority Date: 05/31/2007
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Platt, John C., Basu, Sumit, Nguyen, Patrick, Gupta, Surabhi, Mahajan, Milind V.
• Application Number:
  US11/756,059
• Publication Number:
  US 20080300872A1
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/64 Browsing; Visualisation therefor generation of a list or set of audio data G06F16/638
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

Providing for browsing a summary of content formed of keywords that can scale to a user-defined level of detail is disclosed herein. Components of a system can include a summarization component that extracts keywords related to the content and associates the keywords with portions thereof, and a zooming component that displays a number of keywords based on a keyword/keyphrase relevance rank and a zoom factor. Additionally, a speech to text component can translate speech associated with the content into text, wherein the keywords are extracted from the translated text. Consequently, the claimed subject matter can present a variable hierarchy of keywords to form a scalable summary of such recorded content.

Claims

1. A system that facilitates review of content, comprising:

a browsing interface that receives text associated with or descriptive of audio or visual content, or both, or combinations thereof, and

a summarization component that extracts a plurality of keywords related to the received text, and creates a summarization hierarchy of the audio or visual content, or both, by presenting dynamically adjustable portions of the extracted keywords at the browsing interface.

2. The system of claim 1, further comprising a zoom component that adjusts the presentation of portions of the extracted keywords based on a keyphrase relevance rank and a zoom factor to reveal different levels of detail with respect to the audio or visual content, or both.

3. The system of claim 2, the zoom component displays multiple keywords as a function of an amount of graphical space associated with the zoom factor available to render keywords, and a number of keywords that fit within the graphical space in an order related to the keyphrase relevance rank.

4. The system of claim 1, comprising a temporal sequence component that structures display of one or more of the plurality of keywords according to a temporal occurrence of such keywords within the received text or the audio or visual content.

5. The system of claim 1 further comprising a playback component that plays portions of the audio or visual content, or both, based on selection of an associated keyword.

6. The system of claim 1, further comprising a topic segmentation component that identifies one or more topics within received text, and groups one or more of the plurality of keywords as a function of relationship to the one or more topics.

7. The system of claim 1, further comprising a context component that presents additional surrounding text for one or more of the plurality of keywords to provide context for the keywords.

8. The system of claim 1, further comprising a turn recognition component that groups text associated with the audio or visual content, or both, as a function of contiguous segments spoken by a single speaker.

9. The system of claim 1, further comprising an external application, the keyphrase relevance rank associated with one or more of the plurality of keywords is modified based at least in part on a context relevant to the external application.

10. The system of claim 2, the keyphrase relevance rank is based at least in part on non-verbal cues, speaker turn information, visual cues, TFIDF score, or textual context, or combinations thereof.

11. The system of claim 1, further comprising a speech recognition component, wherein at least a portion of the received text is translated from speech into text by the speech recognition component.

12. A method for providing scalable summaries of recorded content comprising:

analyzing content to identify speech or distinctive audio patterns, contained therein;

identifying one or more keywords associated with the speech or distinctive audio patterns; and

presenting at least one of the one or more keywords based on a relevance rank in relation to a scale factor.

13. The method of claim 12, further comprising extracting the keywords from the content based at least in part on relevance to events within the content.

14. The method of claim 12, further comprising mapping a portion of recorded content to the one or more related keywords.

15. The method of claim 14, further comprising playing the portion of recorded content if one or more of the related keywords mapped to the portion are selected, and graphically distinguishing keywords that are relevant to concurrently played portions of the recorded content.

16. The method of claim 12, the keyword rank is based at least in part on non-verbal cues, a TFIDF factor associated with the keyword, visual cues, speaker turn information including a number of speaker turns containing the keyword, or combinations thereof.

17. The method claim 12, further comprising segmenting the speech or distinctive audio patterns, or both, into one or more topics.

18. A system that facilitates review of audio or visual content, comprising:

means for visually representing portions of content with keywords related to translated speech, key-sounds associated with audio, or both; and

means for displaying a number of keywords representing portions of content based on a relevance rank associated with each of the number of keywords and a user-defined scale factor.

19. The system of claim 18, further comprising means for transcribing spoken words contained on storage media into text.

20. The system of claim 18, further comprising means for dynamically increasing or decreasing a display of keywords in response to increasing and decreasing the user-defined scale factor.