MULTI-MODAL COLLABORATIVE WEB-BASED VIDEO ANNOTATION SYSTEM

• US 20130145269A1
• Filed: 09/26/2012
• Published: 06/06/2013
• Est. Priority Date: 09/26/2011
• Status: Active Grant

• Current Assignee: University of North Carolina At Charlotte (University of North Carolina System)
• Original Assignee: University of North Carolina At Charlotte (University of North Carolina System)
• Inventors: LATULIPE, Celine, SINGH, Vikash, WILSON, David, HUSKEY, Sybil
• Granted Patent: US 9,354,763 B2
• Time in Patent Office: Days
• US Class Current: 715/720
• CPC Class Codes: G06F 16/7867 using information manually generated, e.g. tags, keywords, comments, title and artist information, manually generated time, location and usage information, user ratings
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06Q 10/101 Collaborative creation, e.g. joint development of products or services
• CPC Class Codes: G06Q 10/103 Workflow collaboration or project management
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8547 involving timestamps for synchronizing content

Abstract

A video annotation interface includes a video pane configured to display a video, a video timeline bar including a video play-head indicating a current point of the video which is being played, a segment timeline bar including initial and final handles configured to define a segment of the video for playing, and a plurality of color-coded comment markers displayed in connection with the video timeline bar. Each of the comment markers is associated with a frame or segment of the video and corresponds to one or more annotations for that frame or segment made by one of a plurality of users. Each of the users can make annotations and view annotations made by other users. The annotations can include annotations corresponding to a plurality of modalities, including text, drawing, video, and audio modalities.

Claims

1. A method comprising:

(a) displaying, to a user via an electronic display associated with an electronic device, a login interface;

(b) receiving, at the electronic device from the user, input corresponding to login credentials of a user account of the user;

(c) communicating, from the electronic device, data corresponding to the login credentials;

(d) displaying, to the user via the electronic display, a video selection interface configured to allow the user to a select a video for viewing;

(e) receiving, at the electronic device from the user, input corresponding to selection of a video for viewing;

(f) communicating, from the electronic device, data corresponding to the selected video;

(g) receiving, at the electronic device,

(i) data corresponding to the video, and

(ii) data corresponding to a plurality of comments associated with the video, each of the plurality of comments being associated with a particular user account of a plurality of user accounts;

(h) displaying, to the user via the electronic display associated with the electronic device, a video annotation interface comprising

(i) a video pane configured to display the video,

(ii) a video timeline bar including a video play-head indicating a current point of the video which is being played,

(iii) a segment timeline bar including initial and final handles configured to define a segment of the video for playing,

(iv) a plurality of comment markers displayed in connection with the video timeline bar, each of the plurality of comment markers corresponding to one of the plurality of comments associated with the video,

(v) a comment display pane displaying text corresponding to at least some of the plurality of comments associated with the video, and

(vi) a comment button configured to allow the user to add a comment to the video; and

(i) receiving, at the electronic device from the user, input corresponding to selection of a displayed comment marker; and

(j) in response to receiving input corresponding to selection of a displayed comment marker,

(i) automatically moving the video play-head to a point on the video determined based on the selected comment marker, and displaying, in the video pane, a frame corresponding to the position of the video play-head, and

(ii) displaying, in the video pane overlaid over the displayed frame of the video, one or more annotations associated with the comment marker.

2. The method of claim 1, wherein automatically moving the video play-head to a point on the video determined based on the selected comment marker comprises automatically moving the video play-head to a point of the video a predetermined period of time prior to a point on the video associated with the comment marker.

3. The method of claim 2, wherein the predetermined period of time is two seconds.

4. The method of claim 1, wherein in addition to automatically moving the video play-head to a point on the video determined based on the selected comment marker, the method includes automatically defining a segment corresponding to the selected comment marker utilizing the segment timeline bar by moving the initial handle of the segment timeline bar to a beginning point corresponding to the point on the video determined based on the selected comment marker, and automatically moving the final handle of the segment timeline bar to an end point corresponding to a second point on the video determined based on the selected comment.

5. The method of claim 4, wherein the method further includes

(a) receiving, at the electronic device from the user, input corresponding to engagement of a play button of the video annotation interface;

(b) repeatedly playing the video in the video pane in a loop from a frame corresponding to the position of the initial handle to a frame corresponding to the position of the final handle.

9. The method of claim 4, wherein the received data corresponding to a plurality of comments associated with the video includes data corresponding to a particular segment comment associated with the later-selected comment marker, and wherein the data corresponding to the particular segment comment includes an identification of a starting point of a segment of the video associated with the particular segment comment.

10. The method of claim 9, wherein the data corresponding to the particular segment comment includes an identification of a length of the segment comment.

11. The method of claim 9, wherein the data corresponding to the particular segment comment includes an identification of an end point of a segment of the video associated with the particular segment comment.

12. The method of claim 1, wherein the displayed video annotation interface further comprises a playback speed slider configured to allow the user to vary the speed of playback of the video in the video pane.

17. The method of claim 1, wherein the electronic display is a touchscreen display, and wherein at least one of the steps of receiving, at the electronic device from the user, input, comprises receiving input from the user input via the touchscreen display.

18. The method of claim 1, wherein the method further includes, in response to receiving input corresponding to selection of a displayed comment marker, highlighting a portion of the video displayed in the video pane that corresponds to a focus area associated with a comment corresponding to the comment marker.

19. The method of claim 1, wherein the method further includes, in response to receiving input corresponding to selection of a displayed comment marker, zooming in on a portion of the video displayed in the video pane that corresponds to a zoom area associated with a comment corresponding to the comment marker.

20. The method of claim 1, wherein the method further includes

(a) receiving, at the electronic device from the user, input corresponding to selection of a baseline definition tool;

(b) receiving, at the electronic device from the user, input corresponding to drawing a baseline line segment on the video pane;

(c) receiving, at the electronic device from the user, input representing a baseline measurement for the user-drawn baseline line segment;

(d) receiving, at the electronic device from the user, input corresponding to drawing a measurement line segment on the video pane;

(e) automatically determining a measurement for the measurement line segment based on

(i) a length of the user-drawn baseline line segment,

(ii) the user input representing a baseline measurement for the user-drawn baseline line segment, and

(iii) a length of the user-drawn measurement line segment;

(f) displaying, to the user via the electronic display, the automatically determined measurement for the measurement line segment.

21. The method of claim 1, wherein each of the plurality of comment markers is color coded based on the particular user account associated with a comment the comment marker is associated with.

27. The method of claim 1, wherein the one or more annotations includes annotations in two or more of the following modalities:

text, drawing, video, audio.

6-8. -8. (canceled)

13-16. -16. (canceled)

22-26. -26. (canceled)

28-38. -38. (canceled)

39. A method comprising:

(a) displaying, to a user via an electronic display associated with an electronic device, a login interface;

(b) receiving, at the electronic device from the user, input corresponding to login credentials of a user account of the user;

(c) communicating, from the electronic device, data corresponding to the login credentials;

(d) displaying, to the user via the electronic display, a video selection interface configured to allow the user to a select a video for viewing;

(e) receiving, at the electronic device from the user, input corresponding to selection of a video for viewing;

(f) communicating, from the electronic device, data corresponding to the selected video;

(g) receiving, at the electronic device,

(i) data corresponding to the video, and

(ii) data corresponding to a plurality of comments associated with the video, each of the plurality of comments being associated with a particular user account of a plurality of user accounts;

(h) displaying, to the user via the electronic display associated with the electronic device, a video annotation interface comprising

(i) a video pane configured to display the video,

(ii) a video timeline bar including a video play-head indicating a current point of the video which is being played,

(iii) a segment timeline bar including initial and final handles configured to define a segment of the video for playing,

(iv) a plurality of comment markers displayed in connection with the video timeline bar, each of the plurality of comment markers corresponding to one of the plurality of comments associated with the video,

(v) a comment display pane displaying text corresponding to at least some of the plurality of comments associated with the video, and

(vi) a comment button configured to allow the user to add a comment to the video; and

(i) receiving, at the electronic device from the user, input corresponding to selection of a particular portion of the video;

(j) receiving, at the electronic device from the user, input corresponding to engagement of the comment button;

(k) in response to receiving input corresponding to engagement of the comment button, displaying, to the user via the electronic display associated with the electronic device, a comment interface;

(l) receiving, at the electronic device from the user, input corresponding to one or more desired annotations; and

(m) in response to receiving input corresponding to one or more desired annotations,

(i) associating the input one or more annotations with the selected particular portion of the video,

(ii) updating the video annotation interface so that the plurality of comment markers displayed in connection with the video timeline bar includes a new comment marker indicating a point on the video timeline bar corresponding to the particular portion of the video, and

(iii) displaying an indication of the input one or more annotations overlaid over the video in the video pane.

45. The method of claim 39, wherein the particular portion of the video is a segment.

51. The method of claim 39, wherein receiving, at the electronic device from the user, input corresponding to selection of a particular portion of the video comprises receiving input corresponding to selection of a segment that includes input corresponding to moving the initial handle of the segment timeline bar and moving the final handle of the segment timeline bar.

40-44. -44. (canceled)

46-50. -50. (canceled)

52-61. -61. (canceled)

62. A method comprising:

(a) displaying, to a user via an electronic display associated with an electronic device, a login interface;

(b) receiving, at the electronic device from the user, input corresponding to login credentials of a user account of the user;

(c) communicating, from the electronic device, data corresponding to the login credentials;

(d) displaying, to the user via the electronic display, a video selection interface configured to allow the user to a select a video for viewing;

(e) receiving, at the electronic device from the user, input corresponding to selection of a video for viewing;

(f) communicating, from the electronic device, data corresponding to the selected video;

(g) receiving, at the electronic device,

(i) data corresponding to the video, and

(ii) data corresponding to a plurality of comments associated with the video, each of the plurality of comments being associated with a particular user account of a plurality of user accounts;

(h) displaying, to the user via the electronic display associated with the electronic device, a video annotation interface comprising

(i) a video pane configured to display the video,

(ii) a video timeline bar including a video play-head indicating a current point of the video which is being played,

(iii) a segment timeline bar including initial and final handles configured to define a segment of the video for playing,

(iv) a plurality of comment markers displayed in connection with the video timeline bar, each of the plurality of comment markers corresponding to one of the plurality of comments associated with the video,

(v) a comment display pane displaying text corresponding to at least some of the plurality of comments associated with the video, and

(vi) a comment button configured to allow the user to add a comment to the video; and

(i) receiving, at the electronic device from the user, input corresponding to engagement of a play button of the video annotation interface;

(j) playing, in the video pane, the video;

(k) receiving, at the electronic device from the user, input corresponding to engagement of the comment button;

(l) in response to receiving input corresponding to engagement of the comment button, pausing playback of the video, and displaying, to the user via the electronic display associated with the electronic device, a comment interface;

(m) receiving, at the electronic device from the user, input corresponding to one or more desired annotations; and

(n) in response to receiving input corresponding to one or more desired annotations,

(i) associating the input one or more annotations with the selected particular portion of the video,

(ii) updating the video annotation interface so that the plurality of comment markers displayed in connection with the video timeline bar includes a new comment marker indicating a point on the video timeline bar corresponding to the particular portion of the video, and

(iii) displaying an indication of the input one or more annotations overlaid over the video in the video pane.

63. The method of claim 62, wherein the method further includes loading, at the electronic device, a key code library representing an association of keys with one or more respective desired annotations, and wherein receiving, at the electronic device from the user, input corresponding to one or more desired annotations comprises receiving input corresponding to a particular key which is associated with one or more respective desired annotations in the key code library.

64-67. -67. (canceled)