Annotating video intervals

• US 9,684,644 B2
• Filed: 12/30/2011
• Issued: 06/20/2017
• Est. Priority Date: 02/19/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Badoiu, Mihai, Muthukrishnan, Shanmugavelayutham
• Primary Examiner(s): Tran, Quoc A
• Application Number:
  US13/341,131
• Publication Number:
  US 20120102387A1
• Time in Patent Office: 1,999 Days
• Field of Search: 715230, 715723-732, 386 52, 725 37-109
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/73 Querying
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A system and method is provided for managing and displaying annotations in online hosted videos. To determine related annotations associated with a video, a plurality of annotations for one or more intervals of the video are collected and organized into a plurality of groups. Each group includes annotations for intervals of the video that are similar to each other. A group having related annotations is identified, and an annotated clip of the video is formed based upon the intervals in the group. This process can be used to determine any number of groups in a given video having related annotations, forming the respective annotated intervals within a single video, and can also identify and organize annotated intervals within a large number of different videos.

Claims

1. A method for managing annotations associated with a video, the method comprising:

receiving from viewers of a video, a plurality of annotations for each of a plurality of intervals of the video, each interval having a start time and an end time, a content of each annotation of a plurality of the annotations including a plurality of words;

forming a plurality of annotation groups by clustering the annotations based on the start times and the end times of the intervals;

determining a first annotation group of the plurality of annotations groups having related annotations based at least in part on a comparison of the content of each annotation to determine that the related annotations have at least one identical or similar word;

forming a first annotated clip of the video based upon the intervals in the first annotation group; and

forming a synoptic annotation for the first annotated clip of the video based on the related annotations in the first annotation group, the synoptic annotation comprising one or more words selected from the related annotations in the first annotation group, the one or more words summarizing the content of the related annotations in the first annotation group.

2. The method of claim 1, further comprising:

receiving a search query including one or more search terms;

responsive to receiving the search query, determining that the synoptic annotation matches the one or more search terms; and

displaying the identified synoptic annotation and an identifier of the video associated with the identified synoptic annotation.

3. The method of claim 1, wherein forming the plurality of annotation groups comprises clustering the annotations based on a timestamp associated with each annotation.

4. The method of claim 1, wherein forming the plurality of annotation groups comprises:

identifying a plurality of scenes in the video; and

associating each annotation group with one or more scenes.

5. The method of claim 1, wherein forming the plurality of annotation groups comprises:

identifying a plurality of features in the video; and

associating each annotation group with a set of one or more features in the video.

6. The method of claim 1, further comprising:

assigning a weight to each annotation based on whether the annotation was provided by a unique user; and

determining the first annotation group having related annotations based on the assigned weights of the annotations.

7. The method of claim 1, wherein forming the plurality of annotation groups comprises determining a predetermined number of annotated clips permitted based on a number of indications of annotated clips being visually distinguished on a timeline corresponding to the video.

8. The method of claim 1, wherein forming the plurality of annotation groups comprises determining an amount of time between the intervals with which the annotations are associated.

9. The method of claim 1, further comprising:

displaying an indication of the first annotated clip on a timeline associated with the video.

10. The method of claim 1, further comprising:

determining a second annotation group having related annotations; and

forming a second annotated clip based on the intervals in the second annotation group; and

ranking the first annotated clip and the second annotated clip based on the annotations in each annotation group.

11. A system for managing annotations associated with a video comprising:

a computer processor; and

a non-transitory computer-readable storage medium storing computer program modules configured to execute on the computer processor, the computer program modules comprising;

an extraction module, adapted to receive from viewers of a video, a plurality of annotations for each of a plurality of intervals of the video, each interval having a start time and an end time, a content of each annotation of a plurality of the annotations including a plurality of words;

a grouping module, adapted to form a plurality of annotation groups by clustering the annotations based on the start times and the end times of the intervals;

an annotation determination module, adapted to;

determine a first annotation group of the plurality of annotations groups having related annotations based at least in part on a comparison of the content of each annotation to determine that the related annotations have at least one identical or similar word; and

form an annotated clip of the video based upon the intervals in the first annotation group; and

a synoptic annotation module, adapted to form a synoptic annotation for the annotated clip of the video based on the related annotations in the first annotation group, the synoptic annotation comprising one or more words selected from the related annotations in the first annotation group, the one or more words summarizing the content of the related annotations in the first annotation group.

12. The system of claim 11, wherein forming the plurality of annotation groups comprises clustering the annotations based on a timestamp associated with each annotation.

13. The system of claim 11, wherein forming the plurality of annotation groups comprises:

identifying a plurality of scenes in the video; and

associating each annotation group with one or more scenes.

14. The system of claim 11, wherein forming the plurality of annotation groups comprises:

identifying a plurality of features in the video; and

associating each annotation group with a set of one or more features in the video.

15. A computer program product having a non-transitory computer readable storage medium and including computer executable code for managing annotations associated with a video, the code adapted to perform steps comprising:

receiving from viewers of a video, a plurality of annotations for each of a plurality of intervals of the video, each interval having a start time and an end time, a content of each annotation of a plurality of the annotations including a plurality of words;

forming a plurality of annotation groups by clustering the annotations based on the start times and the end times of the intervals;

determining a first annotation group of the plurality of annotations groups having related annotations based at least in part on a comparison of the content of each annotation, the related annotations having at least one identical or similar word;

forming a first annotated clip of the video based upon the intervals in the first annotation group; and

forming a synoptic annotation for the first annotated clip of the video based on the related annotations in the first annotation group, the synoptic annotation comprising one or more words selected from the related annotations in the first annotation group, the one or more words summarizing the content of the related annotations in the first annotation group.

16. The computer program product of claim 15, wherein forming the plurality of groups comprises clustering the annotations based on a timestamp associated with each annotation.

17. The computer program product of claim 15, wherein forming the plurality of groups comprises:

identifying a plurality of scenes in the video; and

associating each group with one or more scenes.

18. The computer program product of claim 15, wherein forming the plurality of groups comprises:

identifying a plurality of features in the video; and

associating each group with a set of one or more features in the video.