Intelligent Video Summaries in Information Access

• US 20080097970A1
• Filed: 10/18/2006
• Published: 04/24/2008
• Est. Priority Date: 10/19/2005
• Status: Active Grant

• Current Assignee: Microsoft Corporation
• Original Assignee: Fast Search & Transfer ASA (Microsoft Corporation)
• Inventors: Arentz, Will, Olstad, Bjorn
• Granted Patent: US 8,296,797 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/025 Phonemes, fenemes or fenones being the recognition units
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04N 21/43 Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware real-time communication protocols in data switching networks H04L65/00
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/454 Content or additional data filtering, e.g. blocking advertisements
• CPC Class Codes: H04N 21/462 Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/472 End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content end-user interfaces for retrieving video data from a database G06F16/739; network services for supporting unicast streaming H04L65/612
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/482 : End-user interface for program selection broadcast systems using EPGs H04H60/72
• CPC Class Codes: H04N 21/4828 : for searching program descriptors retrieval of video data G06F16/739

Abstract

In a method for displaying video data within result presentations in information access or search systems, compressed thumbnails are computed for videos in a result page and applied to the result page. An end user is enabled to select a video in the result page and activate streaming of a video summary within the context of the associated result page.

Claims

1. A method for displaying video data within result presentations in information access systems or information search systems wherein the method comprises steps for

applying a query to a set of videos, identifying matching videos,

producing a result page with at least one of the most relevant videos, and

wherein the method is characterized by further steps for pre-computing compressed thumbnails for the videos in the result page, applying one or more pre-computed compressed thumbnails to the result pages,

enabling a user or client selection of a video in the result page, and

activating streaming of a video summary within the context of an associated result page.

2. A method according to claim 1,

characterized by the searched videos containing at least one of video files, webinars, streaming video sources or live video broadcasts.

3. A method according to claim 1,

characterized by matching of a query being based on one of textual metadata associated with the video, structured metadata associated with the video including at least one of video size, video length, video quality or video format, text extracted from an audio track by speech-to-text analysis, or phonetic transcription extracted from the audio track;

or combinations thereof

4. A method according to claim 1,

characterized by optimizing the pre-computed compressed thumbnails to the display capabilities of a client device used to view the search results.

5. A method according to claim 1,

characterized by using at least one still frame from the associated video in the pre-computed compressed thumbnails.

6. A method according to claim 5,

characterized by selecting the at least one still frame in response to the occurrence of query terms inside the audio track of the video.

7. A method according to claim 1,

characterized by said user selecting including either hoovering, mouse over, scrolling or clicking on a video in the result set.

8. A method according to claim 1,

characterized by the result page being a web page viewed inside a browser.

9. A method according to claim 1 characterized by using a wireless mobile device as a client device.

10. A method according to claim 9,

characterized by presenting the result page by scrolling through the videos thereof individually in full screen modus.

11. A method according to claim 1,

characterized by optimizing the video summary to a display capability of a client device used to view the search results.

12. A method according to claim 1,

characterized by constructing the video summary by combining at least one time sequence from an original video of the result page.

13. A method according to claim 12,

characterized by selecting the least one time sequence in response to one of the occurrence of query terms inside the audio track of the video, scene changes in the video, motion analysis in the video, or audio analysis;

or

combinations thereof.

14. A method according to claim 1,

characterized by constructing an audio track of the video summary by combining audio tracks associated with the at least time sequence.

15. A method according to claim 1,

characterized by caching the video summaries by the client after streaming, such that subsequent replays do not require additional streaming of video data.

16. A method according to claim 1,

characterized by encoding video summaries with flash technology.

17. A method according to claim 1 wherein the playback of video summaries are done in slow motion.

18. A method according to claim 1,

characterized by constructing the video summary adaptively such that the streaming requirements match the capabilities of a transfer link to the client performing a video search.

19. A method according to claim 1,

characterized by integrating advertising information inside the video summaries.

20. A method according to claim 19,

characterized by integrating the advertising information by mixing advertising video sequences with selected sequences from original video data.

21. A method according to claim 20,

characterized by including in the video summary one of sponsored links from paid web search engines and which can be activated by an end user during viewing/streaming of the video summary, actionable widgets inside the view of the video summary that can enable actions relating the enc user to an advertiser, or display and means for the end user to buy a multimedia object such as a song or a video;

or combinations thereof.