Media Annotation Visualization Tools and Techniques, and an Aggregate-Behavior Visualization System Utilizing Such Tools and Techniques

• US 20130185657A1
• Filed: 09/23/2011
• Published: 07/18/2013
• Est. Priority Date: 09/23/2010
• Status: Active Grant

• Current Assignee: Carnegie Mellon University, University of Louisville
• Original Assignee: Carnegie Mellon University, University of Louisville
• Inventors: Kaufer, David, Gunawardena, Ananda, Cheek, Alexander R. W., Wolfe, Joanna
• Granted Patent: US 10,061,756 B2
• Time in Patent Office: Days
• US Class Current: 715/753
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

Methods of facilitating the discovery of relationships between/among participants within systems for annotating visual media objects to create new associations, communities, and other relationships. These methods include processing annotations, metadata, and/or other information using one or more aggregate-behavior-visualization algorithms and displaying aggregate-behavior visualizations to users. These visualizations allow users to seek out and discover relationships and provide unique knowledge assets useful for a variety of purposes, including creating smart documents and fostering learning.

Claims

1. A method of facilitating discovery of a relationship in a multi-user annotation system, comprising:

providing a user interface designed and configured to solicit annotations from multiple users for a visual media object;

receiving the annotations from the multiple users via the user interface;

storing the annotations and associated metadata;

processing the annotations and associated metadata with an aggregate-behavior visualization algorithm; and

displaying a first aggregated-behavior visualization as a function of said processing, wherein the first aggregate-behavior visualization is designed and configured to allow a viewer to discover the relationship.

2. A method according to claim 1, wherein said displaying includes displaying a hot-spot visualization.

3. A method according to claim 2, wherein said displaying a hot-spot visualization includes displaying a highlight that highlights a portion of the visual media object.

4. A method according to claim 3, wherein said displaying a hot-spot visualization includes displaying a pop-up annotation window in response to user-selection of the highlight, wherein the pop-up annotation window displays one of the annotations and an identification of the one of the multiple users that provided that one of the annotations.

5. A method according to claim 2, wherein said processing includes performing a semantic analysis of the annotations, and said displaying the hot-spot visualization includes visually identifying hot spots based on the semantic analysis.

6. A method according to claim 5, wherein said displaying the hot-spot visualization includes displaying highlighting on the visual media object that highlights one or more portions of the visual media object.

7. A method according to claim 6, wherein said displaying highlighting includes displaying highlighting of differing visual character as a function of differing semantic content of differing annotations.

8. A method according to claim 7, wherein said displaying highlighting of differing visual character includes displaying transparently colored highlighting of differing colors that represent corresponding respective differing semantic content of the differing annotations.

9. A method according to claim 2, wherein said processing includes performing a tonal analysis of the annotations, and said displaying the hot-spot visualization includes visually identifying hot spots based on the tonal analysis.

10. A method according to claim 9, wherein said displaying the hot-spot visualization includes displaying highlighting on the visual media object that highlights one or more portions of the visual media object.

11. A method according to claim 10, wherein said displaying highlighting includes displaying highlighting of differing visual character as a function of differing tone.

12. A method according to claim 11, wherein said displaying highlighting of differing visual character includes displaying transparently colored highlighting of differing colors that represent corresponding respective differing tones.

13. A method according to claim 2, wherein said processing includes performing a density analysis of the hot spots, and said displaying the hot-spot visualization includes visually identifying hot spots based on the density analysis.

14. A method according to claim 13, wherein said displaying the hot-spot visualization includes displaying highlighting on the visual media object that highlights one or more portions of the visual media object.

15. A method according to claim 14, wherein said displaying highlighting includes displaying highlighting of differing visual character as a function of differing density.

16. A method according to claim 1, wherein the user interface further includes a filter tool that allows any of the multiple users to input a filter parameter, the method further comprising receiving a filter parameter value via the filter tool, and said processing including processing the annotations and associated metadata as a function of the filter parameter value.

17. A method according to claim 16, wherein said receiving the filter parameter value includes receiving user-profile information.

18. A method according to claim 16, wherein said receiving the filter parameter value includes receiving a value that indicates desired similarity of the annotations.

19. A method according to claim 16, wherein the filter parameter is annotator group size.

20. A method according to claim 19, further comprising allowing user-selection between an interest group corresponding to the visual media object and all annotators that have annotated the visual media object.

21. A method according to claim 1, wherein the user interface further includes an annotation rating tool that allows any of the multiple users to rate various ones of the annotations, the method further comprising receiving a rating value via the tool, and said processing including processing the annotations and associated metadata as a function of the rating value.

22. A method according to claim 1, wherein said providing the user interface includes providing a user interface designed and configured to allow the multiple users to input the annotations as free-form comments, and said receiving the annotations includes receiving the free-form comments from the multiple users and linking each of the annotations to a corresponding first portion of the visual media object.

23. A method according to claim 22, wherein said providing the user interface further includes providing the user interface with a breadcrumb feature that allows, for each of the annotations, the corresponding one of the multiple users to link a second portion of the visual media object to that annotation.

24. A method according to claim 1, wherein said providing the user interface includes providing a user interface designed and configured to pose questions and allow the multiple users to input the annotations as answers to the questions, and said receiving the annotations includes receiving the answers from the multiple users.

25. A method according to claim 24, wherein said providing the user interface further includes providing the user interface with a breadcrumb feature that allows, for each of the annotations, the corresponding one of the multiple users to link a second portion of the visual media object to the annotation.

26. A method according to claim 1, wherein said providing the user interface includes providing a user interface designed and configured to allow the multiple users to input the annotations by selection of tags, and said receiving the annotations includes receiving selections of the tags from the multiple users.

27. A method according to claim 1, wherein said providing the user interface includes providing a user interface designed and configured to allow the multiple users to input the annotations by selecting a rating value, and said receiving the annotations includes receiving selections of rating values from the multiple users.

28. A method according to claim 27, wherein said providing a user interface designed and configured to allow the multiple users to input the annotations by selecting a rating value includes providing a slider tool.

29. A method according to claim 1, further comprising displaying a second aggregate-behavior visualization simultaneously with the first aggregate-behavior visualization.

30. A method according to claim 29, wherein said displaying a second aggregate-behavior visualization includes displaying a user-interactive aggregate-behavior visualization.

31. A method according to claim 30, further comprising receiving a user interaction with the user-interactive aggregate-behavior visualization and modifying the first aggregate-behavior visualization as a function of the user interaction.

32. A method according to claim 30, wherein the first aggregate-behavior visualization is highlighting of the visual media object and the second aggregate-behavior visualization is an interactive chart.

33. A method according to claim 1, wherein said providing a user interface includes providing moderator controls that allow a moderator to set a window defining when the multiple users can annotate the visual media object.

34. A method according to claim 1, wherein said providing a user interface include providing moderator controls that allow a moderator to set an access time defining when the multiple users can access the first aggregate-behavior visualization.

35. A method according to claim 1, wherein said processing the annotations includes processing the annotations so as to rank ones of the annotations without receiving user rankings.

36. A method according to claim 1, wherein said displaying a first aggregate-behavior visualization includes displaying a graphic depicting user contributions to the visual media object as percentages.

37. A method according to claim 1, further comprising allowing a first user of the multiple users to discover an interest of second user of the multiple users via the first aggregate-behavior algorithm and to follow the second user within the multi-user annotation system as a function of the discovered interest.

38. A method according to claim 37, further comprising providing a first tool that allows the first user to send a follow request to the second user and a second tool that allows the second user to deny the request.

39. A machine-readable medium containing machine-executable instructions for performing a method of facilitating discovery of a relationship in a multi-user annotation system, the machine-executable instructions comprising:

a first set of machine-executable instructions for providing a user interface designed and configured to solicit annotations from multiple users for a visual media object;

a second set of machine-executable instructions for receiving the annotations from the multiple users via the user interface;

a third set of machine-executable instructions for storing the annotations and associated metadata;

a fourth set of machine-executable instructions for processing the annotations and associated metadata with an aggregate-behavior visualization algorithm; and

a fifth set of machine-executable instructions for displaying a first aggregated-behavior visualization as a function of said processing, wherein the first aggregate-behavior visualization is designed and configured to allow a viewer to discover the relationship.

40. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions includes machine-executable instructions for displaying a hot-spot visualization.

41. A machine-readable medium according to claim 40, wherein said machine-executable instructions for displaying a hot-spot visualization includes machine-executable instructions for displaying a highlight that highlights a portion of the visual media object.

42. A machine-readable medium according to claim 41, wherein said machine-executable instructions displaying a hot-spot visualization includes machine-executable instructions displaying a pop-up annotation window in response to user-selection of the highlight, wherein the pop-up annotation window displays one of the annotations and an identification of the one of the multiple users that provided that one of the annotations.

43. A machine-readable medium according to claim 40, wherein said fourth set of machine-executable instructions includes machine-executable instructions for performing a semantic analysis of the annotations, and said machine-executable instructions for displaying the hot-spot visualization includes machine-executable instructions for visually identifying hot spots based on the semantic analysis.

44. A machine-readable medium according to claim 43, wherein said machine-executable instructions for displaying the hot-spot visualization includes machine-executable instructions for displaying highlighting on the visual media object that highlights one or more portions of the visual media object.

45. A machine-readable medium according to claim 44, wherein said machine-executable instructions for displaying highlighting includes machine-executable instructions for displaying highlighting of differing visual character as a function of differing semantic content of differing annotations.

46. A machine-readable medium according to claim 45, wherein said machine-executable instructions for displaying highlighting of differing visual character includes machine-executable instructions displaying transparently colored highlighting of differing colors that represent corresponding respective differing semantic content of the differing annotations.

47. A machine-readable medium according to claim 40, wherein said fourth set of machine-executable instructions includes machine-executable instructions for performing a tonal analysis of the annotations, and said machine-executable instructions for displaying the hot-spot visualization includes machine-executable instructions for visually identifying hot spots based on the tonal analysis.

48. A machine-readable medium according to claim 47, wherein said machine-executable instructions for displaying the hot-spot visualization includes machine-executable instructions displaying highlighting on the visual media object that highlights one or more portions of the visual media object.

49. A machine-readable medium according to claim 48, wherein said machine-executable instructions for displaying highlighting includes machine-executable instructions for displaying highlighting of differing visual character as a function of differing tone.

50. A machine-readable medium according to claim 49, wherein said machine-executable instructions for displaying highlighting of differing visual character includes machine-executable instructions for displaying transparently colored highlighting of differing colors that represent corresponding respective differing tones.

51. A machine-readable medium according to claim 40, wherein said fourth set of machine-executable instructions includes machine-executable instructions for performing a density analysis of the hot spots, and said machine-executable instructions for displaying the hot-spot visualization includes machine-executable instructions for visually identifying hot spots based on the density analysis.

52. A machine-readable medium according to claim 51, wherein said machine-executable instructions for displaying the hot-spot visualization includes machine-executable instructions for displaying highlighting on the visual media object that highlights one or more portions of the visual media object.

53. A machine-readable medium according to claim 52, wherein said machine-executable instructions for displaying highlighting includes machine-executable instructions for displaying highlighting of differing visual character as a function of differing density.

54. A machine-readable medium according to claim 39, wherein the user interface further includes a filter tool that allows any of the multiple users to input a filter parameter, the machine-readable medium further containing machine-executable instructions for receiving a filter parameter value via the filter tool, and said fourth set of machine-executable instructions including machine-executable instructions for processing the annotations and associated metadata as a function of the filter parameter value.

55. A machine-readable medium according to claim 54, wherein said machine-executable instructions for receiving the filter parameter value includes machine-executable instructions for receiving user-profile information.

56. A machine-readable medium according to claim 54, wherein said machine-executable instructions for receiving the filter parameter value includes machine-executable instructions for receiving a value that indicates desired similarity of the annotations.

57. A machine-readable medium according to claim 54, wherein the filter parameter is annotator group size.

58. A machine-readable medium according to claim 57, further containing machine-executable instructions for allowing user-selection between an interest group corresponding to the visual media object and all annotators that have annotated the visual media object.

59. A machine-readable medium according to claim 39, wherein the user interface further includes an annotation rating tool that allows any of the multiple users to rate various ones of the annotations, the machine-readable medium further containing machine-executable instructions for receiving a rating value via the tool, and said fourth set of machine-executable instructions including machine-executable instructions for processing the annotations and associated metadata as a function of the rating value.

60. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions includes machine-executable instructions for providing a user interface designed and configured to allow the multiple users to input the annotations as free-form comments, and said second set of machine-executable instructions includes machine-executable instructions for receiving the free-form comments from the multiple users and linking each of the annotations to a corresponding first portion of the visual media object.

61. A machine-readable medium according to claim 60, wherein said first set of machine-executable instructions further includes machine-executable instructions for providing the user interface with a breadcrumb feature that allows, for each of the annotations, the corresponding one of the multiple users to link a second portion of the visual media object to that annotation.

62. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions includes machine-executable instructions for providing a user interface designed and configured to pose questions and allow the multiple users to input the annotations as answers to the questions, and said second set of machine-executable instructions includes machine-executable instructions for receiving the answers from the multiple users.

63. A machine-readable medium according to claim 62, wherein said first set of machine-executable instructions further includes machine-executable instructions for providing the user interface with a breadcrumb feature that allows, for each of the annotations, the corresponding one of the multiple users to link a second portion of the visual media object to the annotation.

64. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions includes machine-executable instructions for providing a user interface designed and configured to allow the multiple users to input the annotations by selection of tags, and said second set of machine-executable instructions includes machine-executable instructions for receiving selections of the tags from the multiple users.

65. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions includes machine-executable instructions for providing a user interface designed and configured to allow the multiple users to input the annotations by selecting a rating value, and said second set of machine-executable instructions includes receiving selections of rating values from the multiple users.

66. A machine-readable medium according to claim 65, wherein said machine-executable instructions for providing a user interface designed and configured to allow the multiple users to input the annotations by selecting a rating value includes machine-executable instructions providing a slider tool.

67. A machine-readable medium according to claim 39, further containing machine-executable instructions for displaying a second aggregate-behavior visualization simultaneously with the first aggregate-behavior visualization.

68. A machine-readable medium according to claim 67, wherein said machine-executable instructions for displaying a second aggregate-behavior visualization includes machine-executable instructions for displaying a user-interactive aggregate-behavior visualization.

69. A machine-readable medium according to claim 68, further containing machine-executable instructions for receiving a user interaction with the user-interactive aggregate-behavior visualization and modifying the first aggregate-behavior visualization as a function of the user interaction.

70. A machine-readable medium according to claim 68, wherein the first aggregate-behavior visualization is highlighting of the visual media object and the second aggregate-behavior visualization is an interactive chart.

71. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions includes machine-executable instructions providing moderator controls that allow a moderator to set a window defining when the multiple users can annotate the visual media object.

72. A machine-readable medium according to claim 39, wherein said first set of machine-executable instructions include machine-executable instructions for providing moderator controls that allow a moderator to set an access time defining when the multiple users can access the first aggregate-behavior visualization.

73. A machine-readable medium according to claim 39, wherein said fourth set of machine-executable instructions includes machine-executable instructions for processing the annotations so as to rank ones of the annotations without receiving user rankings.

74. A machine-readable medium according to claim 39, wherein said fifth set of machine-executable instructions includes machine-executable instructions for displaying a graphic depicting user contributions to the visual media object as percentages.

75. A machine-readable medium according to claim 39, further containing machine-executable instructions for allowing a first user of the multiple users to discover an interest of second user of the multiple users via the first aggregate-behavior algorithm and to follow the second user within the multi-user annotation system as a function of the discovered interest.

76. A machine-readable medium according to claim 75, further containing machine-executable instructions for providing a first tool that allows the first user to send a follow request to the second user and a second tool that allows the second user to deny the request.