Annotating video intervals

• US 9,690,768 B2
• Filed: 12/31/2013
• Issued: 06/27/2017
• Est. Priority Date: 02/19/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Badoiu, Mihai, Muthukrishnan, Shanmugavelayutham
• Primary Examiner(s): Tran, Quoc A
• Application Number:
  US14/145,786
• Publication Number:
  US 20140115441A1
• Time in Patent Office: 1,274 Days
• Field of Search: 715230, 715723-732, 386 52, 725 37-109
• CPC Class Codes: G06F 16/244 Grouping and aggregation
• CPC Class Codes: G06F 16/24573 using data annotations, e.g. user-defined metadata
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: G06F 16/73 Querying
• CPC Class Codes: G06F 16/738 Presentation of query results
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 30/00 Computer-aided design [CAD]
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A system and method is provided for managing and displaying annotations in online hosted videos. To determine related annotations associated with a video, a plurality of annotations for one or more intervals of the video are collected and organized into a plurality of groups. Each group includes annotations for intervals of the video that are similar to each other. A group having related annotations is identified, and an annotated clip of the video is formed based upon the intervals in the group. This process can be used to determine any number of groups in a given video having related annotations, forming the respective annotated intervals within a single video, and can also identify and organize annotated intervals within a large number of different videos.

Claims

1. A method for annotating a video, comprising:

displaying by a computer on a device a video player graphical user interface, the user interface including a display area for presenting a video and associated control buttons, the video including a plurality of annotated intervals;

displaying, by the computer, a timeline associated with the video;

displaying, by the computer in the display area, a frame of the video ;

receiving, by the computer, a user selection of a region of the displayed frame;

responsive to receiving the selection of the region, displaying, by the computer in the displayed frame, an annotation definition image indicating the selected region;

receiving, by the computer, a user selection of a control button to create an annotation;

providing, by the computer, a display area for receiving user input of annotation content;

receiving, by the computer in the display area, user input of annotation content;

associating the received annotation content with the displayed frame of the video and the selected region to generate an annotated video frame;

ranking the plurality of annotated intervals based on a number of annotations associated with each annotated interval;

indicating on the timeline a marker associated with the annotated video frame;

while displaying the timeline, displaying an annotated thumbnail associated with the annotated video frame in the user interface responsive to a user interaction with the marker; and

displaying indications of ranked orders with markers associated with each of the annotated intervals on the timeline.

2. The method of claim 1 further comprising:

receiving, by the computer, a user selection of a control button to search for an annotation;

receiving, by the computer from the user, a search query;

receiving indicia of at least one video having annotation content matching the search query; and

displaying the received indicia on the display device.

3. The method of claim 2 further comprising:

receiving, by the computer, a selection of indicia of one of the videos having the matching annotation content; and

displaying, by the computer, the indicated video including the matching annotation content.

4. The method of claim 1 further comprising:

receiving, by the computer, a user selection of a control button to edit the annotation; and

receiving, by the computer, user input of revised annotation content.

5. The method of claim 1, wherein displaying the frame of the video further comprises:

playing the video in the user interface;

receiving, during playback of the video, user selection of a control button;

ceasing playback of the video;

selecting a frame of the video responsive to a portion of the video being played back at the time of receiving the user selection; and

displaying the selected frame in the user interface.

6. The method of claim 1 wherein the display area for receiving user input of annotation content is displayed as a pop-up window within the region.

7. The method of claim 1 wherein the video includes a plurality of annotated frames, and the timeline includes a plurality of markers associated with the annotated frames.

8. The method of claim 7 wherein at least one of the annotated markers represents more than one annotated frame of the video.

9. The method of claim 1 further comprising:

playing the video including the annotated video frame in the user interface, and wherein playing the annotated video frame includes displaying the annotation content with the annotated video frame.

10. The method of claim 9 wherein the annotation content is at least one of text superimposed on the annotated video frame, graphics shown alongside the annotated video frame, and audio reproduced simultaneously with the annotated video frame.

11. The method of claim 1, further comprising:

receiving a user input at the timeline indicating an interval of the video, the indicated interval including a plurality of frames;

wherein the displayed frame comprises one or more frames of the plurality of frames, and wherein the marker identifies the interval.

12. A method displaying annotated video, comprising:

displaying by a computer on a device a video player graphical user interface, the user interface including a display area for presenting a video and associated control buttons, the video including a plurality of annotated intervals;

displaying, by the computer, a timeline associated with the video;

displaying, by the computer in the display area, a frame of the video ;

receiving, by the computer, a user selection of a region of the displayed frame;

responsive to receiving the selection of the region, displaying, by the computer in the displayed frame, an annotation definition image indicating the selected region;

receiving, by the computer, a user selection of a control button to create an annotation;

providing, by the computer, a display area for receiving user input of annotation content;

receiving, by the computer in the display area, the user input of annotation content;

associating the received annotation content with the displayed frame of the video and the selected region to generate an annotated video frame;

ranking the plurality of annotated intervals based on a number of annotations associated with each annotated interval;

indicating on the timeline a marker associated with the annotated video frame;

while displaying the timeline, displaying an annotated thumbnail associated with the annotated video frame in the user interface responsive to a user interaction with the marker; and

displaying indications of ranked orders with markers associated with each of the annotated intervals on the timeline.

13. The method of claim 12 further comprising:

receiving, by the computer, a selection of indicia of one of the videos having the matching annotation content; and

displaying, by the computer, the indicated video including the matching annotation content.

14. A computer program product for annotating a video, the computer program product stored on a non-transitory computer-readable medium and including instructions configured to cause a processor to execute steps comprising:

displaying by a computer on a device a video player graphical user interface, the user interface including a display area for presenting a video and associated control buttons, the video including a plurality of annotated intervals;

displaying, by the computer, a timeline associated with the video;

displaying, by the computer in the display area, a frame of the video ;

receiving, by the computer, a user selection of a region of the displayed frame;

responsive to receiving the selection of the region, displaying, by the computer in the displayed frame, an annotation definition image indicating the selected region;

receiving, by the computer, a user selection of a control button to create an annotation;

providing, by the computer, a display area for receiving user input of annotation content;

receiving, by the computer in the display area, the user input of annotation content;

associating the annotation content with the displayed frame of the video and the selected region to generate an annotated video frame;

ranking the plurality of annotated intervals based on a number of annotations associated with each annotated interval;

indicating on the timeline a marker associated with the annotated video frame;

while displaying the timeline, displaying an annotated thumbnail associated with the annotated video frame in the user interface responsive to a user interaction with the marker; and

displaying indications of ranked orders with markers associated with each of the annotated intervals on the timeline.

15. The computer program product of claim 14 further comprising:

receiving, by the computer, a user selection of a control button to edit the annotation; and

receiving, by the computer, user input of revised annotation content.

16. The computer program product of claim 14, wherein displaying the frame of the video further comprises:

playing the video in the user interface;

receiving, during playback of the video, user selection of a control button;

ceasing playback of the video;

selecting a frame of the video responsive to a portion of the video being played back at the time of receiving the user selection; and

displaying the selected frame in the user interface.

17. The computer program product of claim 14 wherein the video includes a plurality of annotated frames, and the timeline includes a plurality of annotated markers associated with the annotated frames, at least one of the annotated markers representing more than one annotated frame.

18. The computer program product of claim 14, further comprising:

playing the video including the annotated video frame in the user interface, and wherein playing the annotated video frame includes displaying the annotations with the annotated video frame.

19. The computer program product of claim 18 wherein the annotations include at least one of text superimposed on the annotated video frame, graphics shown alongside the annotated video frame, and audio reproduced simultaneously with the annotated video frame.