Various methods and apparatus for moving thumbnails with metadata

• US 8,196,045 B2
• Filed: 01/23/2007
• Issued: 06/05/2012
• Est. Priority Date: 10/05/2006
• Status: Active Grant

• Current Assignee: RhythmOne LLC (Nexxen International Ltd.)
• Original Assignee: Blinkx Incorporated (Nexxen International Ltd.)
• Inventors: Chandratillake, Suranga, Scheybeler, Matthew, Wilmer, Adam
• Primary Examiner(s): ABDUL-ALI, OMAR R
• Application Number:
  US11/657,227
• Publication Number:
  US 20080086688A1
• Time in Patent Office: 1,960 Days
• Field of Search: 715/719, 715/838, 715/764, 715/862
• US Class Current: 715/719
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Various methods, apparatuses, and systems are described for a moving thumbnail generator. The moving thumbnail generator generates one or more moving thumbnails that are tagged and time-stamped to arbitrary video trigger events that take place in an associated original video file. Each of the moving thumbnails has two or more moving frames derived from its associated original video file at a portion in the original video file that is tagged and time-stamped to one or more relevant video trigger events in order for the generated moving thumbnail to summarize a visual content of the associated original piece of video file to a user. The tag carries information about content that takes place in the original video file and metadata about that content including a time reference to frames that are contextually relevant to the reason why a viewer of the thumbnail might be interested in viewing that thumbnail.

Claims

1. An apparatus, comprising:

a moving thumbnail generator hosted on a server to generate one or more moving thumbnails that are tagged and time-stamped to arbitrary video trigger events that take place in an associated original video file, wherein each of the moving thumbnails comprises two or more frames derived from its associated original video file at a portion in the original video file that is tagged and time-stamped to one or more relevant video trigger events in order for the generated moving thumbnail to summarize a visual content of the associated original video file to a viewer on a display screen of a client device and a tag included with the moving thumbnail carries information about content that takes place in the original video file and metadata about that content including a time reference to frames that are contextually relevant to a reason why the viewer of the moving thumbnail might be interested in viewing the original video file, and wherein each moving thumbnail has a relational link back to the original video file associated with the moving thumbnail that can be used to retrieve the original video file, where the moving thumbnails comprising two or more frames derived from its associated original video file are images displayed within the thumbnail'"'"'s coordinates on the display screen and the images progress through a sequence of images over time within the same coordinates on the display screen, wherein a comparator is used to compare the moving thumbnails with the tagged and time-stamped arbitrary video trigger events that take place in the associated original video file to a reason why the viewer of the moving thumbnail might be interested in viewing the original video file in order to return as a search result the moving thumbnails having frames of content that are contextually relevant to the reason why the viewer of the moving thumbnail might be interested in viewing the original video file, where the moving thumbnail is configured to cooperate with a data store that contains multiple instances of the moving thumbnail for each original video file.

2. The apparatus of claim 1, wherein the moving thumbnail generator has one or more attribute filters to detect attributes from the original video file under analysis and identify the attributes, an indexer to index the attributes and to assign a time ordered indication with each of the identified attributes, and a tag generator to generate the tag carrying metadata that includes key identified attributes from the original video file and relative weights assigned to each of the key identified attributes.

3. The apparatus of claim 1,

wherein the data store is configured to store the moving thumbnails, wherein each original video file has one or more instances of the moving thumbnail associated with that original video file and each of the one or more instances of the moving thumbnail is tied to a separate video trigger event from that original video file, where the tag included with each instance of the moving thumbnail carries the timestamp and the metadata information about the two or more frames to indicate why they are contextually relevant to the reason why the viewer of the thumbnail might be interested in viewing that thumbnail.

4. The apparatus of claim 3, wherein the one or more moving thumbnails for each original video file contain scenes of multiple frames and each of the scenes is mapped to a time line of identified sub-sections of the original video file.

5. The apparatus of claim 3, wherein the tag that carries information about the content that takes place in the original video file further contains data information that ties the moving thumbnail to the point in time, within the original video file, that the two or more frames were extracted from and any relevant metadata from the video file that occurred in or around that moment in time.

6. The apparatus of claim 1, wherein a graphic user interface displays the moving thumbnails in response to at least one of the following:

query terms supplied by the user;

in response to a theme a web page is displaying; and

in response to the user'"'"'s search history, which suggests that the viewer would be interested in the content of the moving thumbnails.

7. The apparatus of claim 1, wherein the relational link when activated is configured to bring back the original video file and start playing the original video file from a point in the original video file offset from a beginning of the original video file that the moving thumbnail was taken.

8. The apparatus of claim 1, wherein the apparatus is a website hosting a search engine on a computer readable medium that has a user interface, the moving thumbnail generator, and a user'"'"'s query input, wherein the search engine to find one or more video files based on a content of the user'"'"'s query input and the user interface to present a search result of generated moving thumbnails of the one or more video files at a portion in the original video file that is tagged and time-stamped to the one or more video trigger events relevant to the content from the user'"'"'s query input in order for the generated moving thumbnail to summarize a visual content of the original video file to the user, wherein each of the moving thumbnails has an associated tag that contains information about the events that occur in the original video file in a searchable format.

9. The apparatus of claim 8, wherein the user interface presents search results to the content of the user'"'"'s query input as a video wall of multiple moving thumbnails.

10. The apparatus of claim 8, wherein the user interface presents the one or more moving thumbnails for the one or more original video files as search results to be displayed in a piece of long-form video in a grid pattern all at the same time to create a video wall of moving thumbnails.

11. The apparatus of claim 9, wherein the video wall is configured to enlarge a first moving thumbnail when a cursor hovers over the first moving thumbnail.

12. The apparatus of claim 8, wherein the user interface to present one or more selectable icons to act as qualifiers that can modify search parameters associated with the content of the user'"'"'s query input.

13. The apparatus of claim 12, wherein the search parameters associated with the content of the user'"'"'s query are also modified based upon a user'"'"'s profile.

14. The apparatus of claim 8, wherein the search engine uses one or more attribute filters to identify various video trigger events relevant to the content in the original video file.

15. The apparatus of claim 14, wherein the one or more attribute filters include an image analysis tool, a speech to text converter, a scene change detector, and a video optical character recognition, and the video trigger events of the content of the original video file include objects in the video file identified through the image analysis tool, words spoken in the video file identified through the speech to text converter, scene change information detected by the scene change detector, and text appearing in a video identified through the video optical character recognition.

16. The apparatus of claim 8, wherein the search engine has two or more types of search retrieval engines and a query control module to select which type of search engine to use based on at least in part the user'"'"'s query input and qualifiers attached to content of the query itself by the user.

17. The apparatus of claim 8, wherein the search engine has a tracking module to allow the search engine to track a user'"'"'s history of search queries and selected search results over time to automatically modify the search parameters associated with a content of a query supplied to the user'"'"'s query input.

18. The apparatus of claim 8, further comprising:

a data store to store the moving thumbnails, wherein each original video file has one or more moving thumbnails associated with that original video file and each of the one or more moving thumbnails is tied to a separate video trigger event from that original video file; and

a query control module to select a first moving thumbnail displaying frames from a section within the original video file that is most relevant to the content of the query supplied by the user.

19. The apparatus of claim 8, wherein the search engine has a content categorizer to allow a user to select to include or exclude various categories of content from which the search results may come from.

20. The apparatus of claim 8, wherein the relational link when activated is configured to bring back the original video file and start playing the original video file from a point in the original video file offset from a beginning of the original video file that the moving thumbnail was taken.

21. A method, comprising:

generating one or more moving thumbnails from a server that are tagged and time-stamped to arbitrary video trigger events that take place in an associated original video file, wherein each of the moving thumbnails includes two or more frames derived from its associated original video file at a portion in the original video file that is tagged and time-stamped to one or more relevant video trigger events in order for the generated moving thumbnail to summarize a visual content of the associated original piece of video file to a user in a search results portions of a display screen of a client device, where the moving thumbnail is configured to cooperate with a data store that contains multiple instances of the moving thumbnail for each original video file;

creating and including a tag associated with its moving thumbnail to contain information about content that takes place in the original video file and metadata about that content including a time reference to frames that are contextually relevant to a reason why the viewer of the thumbnail might be interested in viewing that moving thumbnail; and

embedding a relational link back to the original video file in or near each moving thumbnail, where the relational link when activated retrieves the original video file, where the moving thumbnails comprising two or more frames derived from its associated original video file are images displayed within the thumbnail'"'"'s coordinates on the display screen and the images progress through a sequence of images over time within the same coordinates on the display screen, wherein a comparator is used to compare the moving thumbnails with the tagged and time-stamped arbitrary video trigger events that take place in the associated original video file to a reason why the viewer of the moving thumbnail might be interested in viewing the original video file in order to return the moving thumbnails having frames of content that are contextually relevant to the reason why the viewer of the moving thumbnail might be interested in viewing the original video file.

22. The method of claim 21, further comprising:

creating a video wall of two or more moving thumbnails from the moving thumbnails returned as search results, which are displayed in a grid pattern all at the same time.