Multi-modal collaborative web-based video annotation system

• US 9,354,763 B2
• Filed: 09/26/2012
• Issued: 05/31/2016
• Est. Priority Date: 09/26/2011
• Status: Active Grant

• Current Assignee: University of North Carolina At Charlotte (University of North Carolina System)
• Original Assignee: University of North Carolina At Charlotte (University of North Carolina System)
• Inventors: Latulipe, Celine, Singh, Vikash, Huskey, Sybil, Wilson, David
• Primary Examiner(s): Tran, Tuyetlien T
• Assistant Examiner(s): Ramsey, Patrick
• Application Number:
  US13/627,899
• Publication Number:
  US 20130145269A1
• Time in Patent Office: 1,343 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A video annotation interface includes a video pane configured to display a video, a video timeline bar including a video play-head indicating a current point of the video which is being played, a segment timeline bar including initial and final handles configured to define a segment of the video for playing, and a plurality of color-coded comment markers displayed in connection with the video timeline bar. Each of the comment markers is associated with a frame or segment of the video and corresponds to one or more annotations for that frame or segment made by one of a plurality of users. Each of the users can make annotations and view annotations made by other users. The annotations can include annotations corresponding to a plurality of modalities, including text, drawing, video, and audio modalities.

Claims

1. A method comprising:

(a) displaying, to a user via an electronic display associated with an electronic device, a login interface;

(b) receiving, at the electronic device from the user, input corresponding to login credentials of a user account of the user;

(c) communicating, from the electronic device, data corresponding to the login credentials;

(d) displaying, to the user via the electronic display, a video selection interface configured to allow the user to a select a video for viewing;

(e) receiving, at the electronic device from the user, input corresponding to selection of a video for viewing;

(f) communicating, from the electronic device, data corresponding to the selected video;

(g) receiving, at the electronic device,

(i) data corresponding to the video, and

(ii) data corresponding to a plurality of point comments associated with the video, each of the plurality of point comments being associated with a single point in time and with a particular user account of a plurality of user accounts,

(iii) data corresponding to a plurality of segment comments associated with the video, each of the plurality of segment comments being associated with a video segment representing a span of time of the video and with a particular user account of the plurality of user accounts;

(h) displaying, to the user via the electronic display associated with the electronic device, a video annotation interface comprising

(i) a video pane configured to display the video,

(ii) a first video timeline bar including a video play-head indicating a current point of the video which is being played,

(iii) a second segment timeline bar disposed below the first video timeline bar, the second segment timeline bar including initial and final handles configured to define a segment of the video for playing,

(iv) a first plurality of point comment markers identifiable as point comment markers by the presence of a first geometric shape displayed in connection with the video timeline bar, each of the first plurality of comment markers corresponding to one of the plurality of point comments associated with the video,

(v) a second plurality of segment comment markers identifiable as segment comment markers by the presence of a second geometric shape displayed in connection with the video timeline bar, each of the second plurality of comment markers corresponding to one of the plurality of segment comments associated with the video,

(vi) a comment display pane displaying text corresponding to at least some of the plurality of comments associated with the video, and

(vii) a comment button configured to allow the user to add a comment to the video; and

(i) receiving, at the electronic device from the user, input corresponding to engagement at a first point on the segment timeline bar;

(j) automatically, in response to receiving the input corresponding to engagement at the first point on the segment timeline bar, moving the initial and final handles of the segment timeline bar to define a first segment of a first length centered around the first point, the first length being a length calculated based on a total length of the video;

(k) receiving, at the electronic device from the user, input corresponding to dragging of the final handle to change the length of the first segment to a second length;

(l) receiving, at the electronic device from the user, input corresponding to dragging of the first segment on the segment timeline bar, and simultaneously moving the initial and final handles while keeping the first segment its current length in response thereto;

(m) receiving, at the electronic device from the user, input corresponding to engagement of the comment button;

(n) in response to receiving input corresponding to engagement of the comment button, displaying, to the user via the electronic display associated with the electronic device, a comment interface;

(o) receiving, at the electronic device from the user, input corresponding to one or more desired annotations; and

(p) in response to receiving input corresponding to one or more desired annotations,

(i) associating the input one or more annotations with the selected first segment of the video,

(ii) updating the video annotation interface so that the plurality of comment markers displayed in connection with the video timeline bar includes a new segment comment marker corresponding to the first segment, and

(iii) displaying an indication of the input one or more annotations overlaid over the video in the video pane.

2. The method of claim 1, wherein the received data corresponding to a plurality of point comments associated with the video includes data corresponding to a particular point comment which includes an identification of a point of the video associated with the comment.

3. The method of claim 2, wherein the identification of a point of the video comprises a timestamp.

4. The method of claim 2, wherein the identification of a point of the video comprises a frame identification.

5. The method of claim 1, wherein the received data corresponding to a plurality of segment comments associated with the video includes data corresponding to a particular segment comment, and wherein the data corresponding to the particular segment comment includes an identification of a starting point of a segment of the video associated with the particular segment comment.

6. The method of claim 5, wherein the data corresponding to the particular segment comment includes an identification of a length of the segment comment.

7. The method of claim 5, wherein the data corresponding to the particular segment comment includes an identification of an end point of a segment of the video associated with the particular segment comment.

8. The method of claim 1, wherein the method further includes loading, at the electronic device, a key code library representing an association of keys with one or more respective desired annotations, and wherein receiving, at the electronic device from the user, input corresponding to one or more desired annotations comprises receiving input corresponding to a particular key which is associated with one or more respective desired annotations in the key code library.

9. The method of claim 1, wherein the electronic display is part of the electronic device.

10. The method of claim 1, wherein the electronic device is a tablet, slate computer, or smartphone.

11. The method of claim 1, wherein the electronic display is connected to the electronic device.

12. The method of claim 1, wherein the electronic display is a touchscreen display, and wherein at least one of the steps of receiving, at the electronic device from the user, input, comprises receiving input from the user input via the touchscreen display.

13. The method of claim 1, wherein the method further includes

(a) receiving, at the electronic device from the user, input corresponding to selection of a baseline definition tool;

(b) receiving, at the electronic device from the user, input corresponding to drawing a baseline line segment on the video pane;

(c) receiving, at the electronic device from the user, input representing a baseline measurement for the user-drawn baseline line segment;

(d) receiving, at the electronic device from the user, input corresponding to drawing a measurement line segment on the video pane;

(e) automatically determining a measurement for the measurement line segment based on

(i) a length of the user-drawn baseline line segment,

(ii) the user input representing a baseline measurement for the user-drawn baseline line segment, and

(iii) a length of the user-drawn measurement line segment;

(f) displaying, to the user via the electronic display, the automatically determined measurement for the measurement line segment.

14. The method of claim 1, wherein each of the plurality of comment markers is color coded based on the particular user account associated with a comment the comment marker is associated with.

15. The method of claim 1, wherein the one or more annotations includes a video annotation.

16. The method of claim 1, wherein the one or more annotations includes an audio annotation.

17. The method of claim 1, wherein the one or more annotations includes a drawing annotation.

18. The method of claim 1, wherein the one or more annotations includes annotations in a plurality of different modalities.