Robust anchoring of annotations to content

• US 20040205542A1
• Filed: 09/07/2001
• Published: 10/14/2004
• Est. Priority Date: 09/07/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bernheim Brush, Alice Jane, Gupta, Anoop, Bargeron, David M.
• Granted Patent: US 7,747,943 B2
• Time in Patent Office: Days
• US Class Current: 715/512
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

Annotations are robustly anchored to content. The robust anchoring allows annotations to be associated with the content independently of the format used to store the content. The annotations can be re-anchored to the content after the content has been modified, including modifications to the format used to store the content.

Claims

1. A system comprising:

an annotation marking engine configured to anchor an annotation to a portion of content, the annotation marking engine including,

a beginning point capture module to capture one or more features describing a beginning point of the portion,

an ending point capture module to capture one or more features describing an ending point of the portion, and

a region capture module to capture one or more features describing the portion, wherein the portion comprises a region between the beginning point and the ending point; and

an annotation locating engine configured to receive the content after the content has been modified such that the portion has been altered, and further configured to identify a portion of the modified content to which the annotation is to be anchored.

2. A system as recited in claim 1, wherein the content is stored in one format prior to being modified and stored in a different format after being modified.

3. A system as recited in claim 1, wherein the annotation locating engine comprises:

a candidate region determination module to identify one or more regions of the modified content to which the annotation may potentially be anchored and to generate a score for each of the one or more regions; and

a score comparator module to determine which of the one or more regions the annotation is to be anchored to based at least in part on the generated score for the regions.

4. A system as recited in claim 3, wherein the annotation locating engine further comprises:

a user interface module to query a user regarding whether at least one of the one or more regions is the region to which the annotation is to be anchored.

5. A system as recited in claim 4, wherein the user interface module is further to display an interface that identifies a single region of the identified one or more regions and allows the user to indicate whether the single region is the region to which the annotation is to be anchored.

6. A system as recited in claim 4, wherein the user interface module is further to display an interface including a region selection input option that allows the user to cycle through displaying different ones of the identified one or more regions.

7. A system as recited in claim 3, wherein the annotation locating engine further comprises:

a preferences module to maintain user preferences regarding one or more threshold values to be used in identifying the portion of the modified content that the annotation is to be anchored to.

8. A system as recited in claim 1, wherein the region capture module is to capture, as the one or more features, an indication of one or more keywords in the portion.

9. A system as recited in claim 8, wherein the region capture module is further to capture, as the one or more features, a distance for each of the one or more keywords from the beginning point to the keyword and a distance from the keyword to the endpoint.

10. A system as recited in claim 1, wherein the content comprises one or more of:

text content, image content, video content, and audio content.

11. A system as recited in claim 1, wherein the annotation includes one or more of:

text content, video content, image content, and audio content.

12. A system as recited in claim 1, wherein the beginning point of the portion, the ending point of the portion, and the portion are explicitly identified by a user.

13. A method of anchoring an annotation to content, the method comprising:

capturing one or more features describing a beginning point of a portion of the content;

capturing one or more features describing an ending point of the portion of the content;

capturing one or more features describing the portion between the beginning point and the ending point; and

associating the annotation with the captured features.

14. A method as recited in claim 13, wherein the captured features are independent of the format in which the content is stored.

15. A method as recited in claim 13, further comprising storing the captured features independently from the content.

16. A method as recited in claim 13, wherein the one or more features describing the beginning point of the portion of content comprises an offset indicating the location of the beginning point relative to a particular point in the content.

17. A method as recited in claim 16, wherein the particular point in the content comprises the beginning of the content.

18. A method as recited in claim 13, wherein the one or more features describing the beginning point of the portion of content comprises one or more characters of the content at the beginning point that either precede the beginning point or succeed the beginning point.

19. A method as recited in claim 13, wherein the one or more features describing the beginning point of the portion of content comprises a hash value generated from hashing one or more characters of the content located at the beginning point.

20. A method as recited in claim 13, wherein the one or more features describing the ending point of the portion of content comprises an offset indicating the location of the ending point relative to a particular point in the content.

21. A method as recited in claim 20, wherein the particular point in the content comprises the beginning of the content.

22. A method as recited in claim 13, wherein the one or more features describing the ending point of the portion of content comprises one or more characters of the content at the ending point that either precede the ending point or succeed the ending point.

23. A method as recited in claim 13, wherein the one or more features describing the ending point of the portion of content comprises a hash value generated from hashing one or more characters of the content located at the ending point.

24. A method as recited in claim 13, wherein the one or more features describing the portion comprises a length of the portion.

25. A method as recited in claim 13, wherein the one or more features describing the portion comprises an indication of one or more keywords in the portion.

26. A method as recited in claim 25, wherein the one or more features describing the portion comprises, for each of the one or more keywords, a distance from the beginning point to the keyword and a distance from the keyword to the endpoint.

27. A method as recited in claim 13, wherein the beginning point of the portion, the ending point of the portion, and the portion are explicitly identified by a user.

28. A method as recited in claim 13, further comprising:

receiving an implicit indication of a region of the content to which the annotation corresponds; and

identifying, based on the implicit indication, the portion of the content to which the annotation corresponds.

29. A method as recited in claim 28, wherein the identifying comprises:

identifying a paragraph of the content in close proximity to a mark made by the user; and

using the paragraph as the portion of the content to which the annotation corresponds.

30. A method as recited in claim 13, wherein the content comprises one or more of:

text content, image content, video content, and audio content.

31. A method as recited in claim 13, wherein the annotation includes one or more of:

text content, video content, image content, and audio content.

32. A method as recited in claim 13, further comprising capturing one or more features describing an intermediate point of a portion of the content.

33. A method as recited in claim 32, wherein the one or more features describing an intermediate point comprise one or more characters of the content that either precede the intermediate point or succeed the intermediate point.

34. A method as recited in claim 32, wherein the one or more features describing an intermediate point comprise hash values generated from hashing one or more characters of the content located at the intermediate point.

35. A method of determining a portion of modified content to which an annotation corresponds, the method comprising:

receiving annotation anchoring information, wherein the annotation anchoring information describes a portion of the content that the annotation corresponded to prior to the portion being modified;

identifying keywords in the modified content;

determining, for a region of the content that potentially corresponds to the annotation, a score for the region based at least in part on the location of the keywords in the content;

repeating the determining for a plurality of identified keywords in the content; and

selecting the region having the best score as the portion of content to which the annotation corresponds.

36. A method as recited in claim 35, further comprising selecting the region having the best score as the portion of content to which the annotation corresponds if the best score exceeds a lower bound, and otherwise selecting no region as the portion of content to which the annotation corresponds.

37. A method as recited in claim 35, farther comprising selecting the region having the best score as the portion of content to which the annotation corresponds without user input if the best score exceeds a threshold value, and otherwise obtaining user input prior to selecting the region having the best score as the portion of content to which the annotation corresponds.

38. A method as recited in claim 37, wherein the user input comprises a user request for a best guess as to the region to which the annotation corresponds.

39. A method as recited in claim 35, further comprising selecting the region having the best score as the portion of content to which the annotation corresponds and stopping the repeating if the best score exceeds a threshold value.

40. A method as recited in claim 35, wherein the content and the modified content are stored in two different formats.

41. A method as recited in claim 35, wherein selecting the region having the best score comprises selecting the region having the highest score.

42. A method as recited in claim 35, wherein determining the score for the region comprises:

identifying each keyword in the annotation anchoring information;

selecting one of the keywords in the annotation anchoring information;

locating the selected keyword in the modified content;

locating one or more additional keywords that are both identified in the annotation anchoring information and that are within a particular distance in the modified content of the selected keyword; and

increasing the score for each of the one or more additional keywords.

43. A method as recited in claim 42, wherein the particular distance comprises twice the length of the portion of the content that the annotation corresponding to prior to the portion being modified.

44. A method as recited in claim 35, wherein determining the score for the region comprises:

selecting a keyword from the annotation anchoring information;

identifying, from the annotation anchoring information, one or more beginning point features for the portion of the content;

locating the selected keyword in the modified content;

locating a point of the modified content that precedes the selected keyword and that matches the one or more beginning point features; and

determining whether to increase the score for the region including the selected keyword and the located point based at least in part on the distance between the located point and the selected keyword.

45. A method as recited in claim 44, wherein determining whether to increase the score for the region comprises determining to increase the score for the region if the distance from the located point to the selected keyword is not greater than a threshold amount.

46. A method as recited in claim 44, wherein the located point comprises an intermediate point.

47. A method as recited in claim 35, wherein determining the score for the region comprises:

selecting a keyword from the annotation anchoring information;

identifying, from the annotation anchoring information, one or more ending point features for the portion of the content;

locating the selected keyword in the modified content;

locating a point of the modified content that succeeds the selected keyword and that matches the one or more ending point features; and

determining whether to increase the score for the region including the selected keyword and the located point based at least in part on the distance between the selected keyword and the located point.

48. A method as recited in claim 47, wherein determining whether to increase the score for the region comprises determining to increase the score for the region if the distance from the selected keyword to the located point is not greater than a threshold amount.

49. A method as recited in claim 47, wherein the located point comprises an intermediate point.

50. A method as recited in claim 35, wherein determining the score for the region comprises:

selecting a keyword from the annotation anchoring information;

identifying, from the annotation anchoring information, one or more beginning point features for the portion of the content;

locating the selected keyword in the modified content;

locating a point of the modified content that precedes the selected keyword and that matches the one or more beginning point features;

checking whether the located point is within a threshold distance of a beginning point of the portion of the content that the annotation corresponded to prior to the portion being modified; and

increasing the score for the region if the located point is within the threshold distance of the beginning point of the portion.

51. A method as recited in claim 35, wherein determining the score for the region comprises:

selecting a keyword from the annotation anchoring information;

identifying, from the annotation anchoring information, one or more beginning point features for the portion of the content;

locating the selected keyword in the modified content;

locating a beginning point of the modified content that precedes the selected keyword and that matches the one or more beginning point features;

identifying, from the annotation anchoring information, one or more ending point features for the portion of the content;

locating an ending point of the modified content that succeeds the selected keyword and that matches the one or more ending point features;

determining a distance between the located beginning point and the located ending point;

checking whether the distance between the located beginning point and the located ending point is greater by more than a threshold amount of a distance between the beginning of the portion of the content that the annotation corresponded to prior to the portion being modified and the ending of the portion of the content that the annotation corresponded to prior to the portion being modified; and

increasing the score for the region if the distance is not greater by more than the threshold amount.

52. A method of determining a portion of modified content to which an annotation corresponds, the method comprising:

receiving annotation anchoring information, wherein the annotation anchoring information describes a portion of the content that the annotation corresponded to prior to the content being modified;

determining one or more regions of the modified content that the annotation potentially corresponds to;

displaying, to a user, an interface that allows the user to select one of the one or more regions as the region to which the annotation corresponds; and

identifying, as the portion of modified content to which the annotation corresponds, the user-selected region.

53. A method as recited in claim 52, wherein displaying the interface comprises displaying an interface that identifies a single region of the one or more regions and allowing the user to indicate whether the single region is the region to which the annotation corresponds.

54. A method as recited in claim 52, wherein displaying the interface comprises displaying an interface including a region selection input that allows the user to cycle through multiple different regions of the modified content that the annotation potentially corresponds to.

55. A method as recited in claim 52, wherein displaying the interface is further comprises displaying, as part of the interface, the content of at least one of the one or more regions.

56. A method as recited in claim 52, further comprising:

generating a score for each of the one or more regions, the score for a region indicating how well the region matches the portion of the content that the annotation corresponded to prior to the content being modified; and

displaying, as part of the interface, the score of at least one of the one or more regions.

57. One or more computer readable media having stored thereon a plurality of instructions that, when executed by one or more processors, causes the one or more processors to perform acts comprising:

receiving an annotation anchor that describes a region of original content to which an annotation corresponds;

receiving a modified version of the original content;

identifying one or more key parts in the annotation anchor;

identifying, based at least in part on the one or more key parts, one or more regions of the modified version to which the annotation potentially corresponds;

generating a score for the one or more regions; and

selecting, as the region of the modified version of the original content to which the annotation corresponds, the identified region having the best score.

58. One or more computer readable media as recited in claim 57, wherein the region of original content has been altered in the modified version.

59. One or more computer readable media as recited in claim 57, wherein the selecting comprises:

checking whether the best score exceeds a threshold value;

automatically selecting the identified region having the best score if the best score exceeds the threshold value; and

otherwise, querying a user for input as to whether the identified region is the region of the modified version to which the annotation corresponds.

60. One or more computer readable media as recited in claim 57, wherein the plurality of instructions further cause the one or more processors to perform acts comprising:

generating a new annotation anchor based on captured features of the identified region; and

using the new annotation anchor as an anchor that describes the region of the modified version of the original content to which the annotation corresponds.

61. One or more computer readable media as recited in claim 57, wherein the key parts are keywords, and wherein the plurality of instructions further cause the one or more processors to perform acts comprising:

identifying one or more synonyms for at least one of the one or more keywords; and

altering the score for a region if at least one of the one or more synonyms is included in the region.

62. One or more computer readable media as recited in claim 57, wherein the key parts are keywords of text content.

63. One or more computer readable media as recited in claim 57, wherein the key parts are shot boundaries of video content.

64. One or more computer readable media as recited in claim 57, wherein the plurality of instructions further cause the one or more processors to perform acts comprising:

detecting multiple identified regions that correspond to the region of original content after being split; and

selecting, as the region of the modified version of the original content to which the annotation corresponds, the multiple identified regions.

65. One or more computer readable media as recited in claim 57, wherein the plurality of instructions further cause the one or more processors to perform acts comprising:

identifying one or more features describing each of one or more intermediate points in the modified version; and

using, when generating the score, at least one of the one or more intermediate points.

66. One or more computer readable media as recited in claim 65, wherein the one or more features describing an intermediate point comprise one or more characters of the modified version that either precede the intermediate point or succeed the intermediate point.

67. One or more computer readable media as recited in claim 65, wherein the one or more features describing an intermediate point comprise hash values generated from hashing one or more characters of the modified content located at the intermediate point.

68. One or more computer readable media as recited in claim 57, wherein generating the score for a region comprises:

identifying each of one or more keywords in the annotation anchor;

selecting one of the keywords in the annotation anchor;

locating the selected keyword in the modified version;

locating one or more additional keywords that are both identified in the annotation anchor and that are within a particular distance of the selected keyword in the modified version; and

increasing the score for each of the one or more additional keywords.

69. One or more computer readable media as recited in claim 57, wherein generating the score for a region comprises:

selecting one of one or more keywords from the annotation anchor;

identifying, from the annotation anchor, one or more beginning point features for the region of the original content;

locating the selected keyword in the modified version;

locating a point of the modified version that precedes the selected keyword and that matches the one or more beginning point features; and

determining whether to increase the score for the region based at least in part on the distance between the located point and the selected keyword.

70. One or more computer readable media as recited in claim 57, wherein generating the score for a region comprises:

selecting one of one or more keywords from the annotation anchor;

identifying, from the annotation anchor, one or more ending point features for the region of the original content;

locating the selected keyword in the modified version;

locating a point of the modified version that succeeds the selected keyword and that matches the one or more ending point features; and

determining whether to increase the score for the region based at least in part on the distance between the selected keyword and the located point.

71. One or more computer readable media as recited in claim 57, wherein generating the score for a region comprises:

selecting one of one or more keywords from the annotation anchor;

identifying, from the annotation anchor, one or more beginning point features for the region of the original content;

locating the selected keyword in the modified version;

locating a point of the modified version that precedes the selected keyword and that matches the one or more beginning point features;

checking whether the located point is within a threshold distance of a beginning point of the region of the original content; and

increasing the score for the region if the located point is within the threshold distance of the beginning point of the region.

72. One or more computer readable media having stored thereon a plurality of instructions that, when executed by one or more processors, causes the one or more processors to perform acts comprising:

receiving an indication of a portion of content to which an annotation corresponds;

capturing one or more features regarding a beginning of the portion, an ending of the portion, and the region between the beginning and ending; and

wherein the one or more features are captured independently of a format used to store the content.

73. One or more computer readable media as recited in claim 72, wherein no modifications are made to the content in order to identify the portion to which the annotation corresponds.

74. One or more computer readable media as recited in claim 72, wherein the plurality of instructions further cause the one or more processors to perform acts comprising:

receiving a modified version of the content; and

determining, based at least in part on the one or more captured features, a portion of the modified version of the content to which the annotation corresponds.