Annotating temporally-dimensioned multimedia content

• US 6,546,405 B2
• Filed: 10/23/1997
• Issued: 04/08/2003
• Est. Priority Date: 10/23/1997
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Gupta, Anoop, Cannon, Anthony W.
• Primary Examiner(s): Kindred, Alford W.
• Application Number:
  US08/956,871
• Publication Number:
  US 20020059342A1
• Time in Patent Office: 1,993 Days
• Field of Search: 707/512, 707/500, 707/501, 707/515, 707/517, 707/520, 707/526, 707/102, 707/104, 395/117, 395/851, 345/302, 345/433, 345/435, 345/475, 345/118, 345/148, 345/326, 345/335, 345/691
• US Class Current: 715/233
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes

Abstract

A human viewing temporally-dimensioned content of a multimedia document through use of a computer can add substantive content to temporally-dimensioned content of the multimedia document to thereby annotate, comment upon, and augment the multimedia document. Thus, a multimedia document becomes a basis for collaborative work. The substantive content added by the viewing user is in the form of a temporal annotation which identifies a particular relative time in temporally-dimensioned content of the multimedia document and which includes user-authored content provided by the viewing user. Display of the multimedia document includes presentation of the temporal annotations created by the user. A graphical user interface enables the user to select a temporal annotation from a list and immediately proceed to presentation of the multimedia document such that temporally-dimensioned content is presented at the particular relative time represented by the selected temporal annotation. In addition, as temporally-dimensioned content of the multimedia document is displayed, temporal annotations are represented in the display of the multimedia document as the particular relative time represented by each temporal annotation is reached. Thus, the user-authored content becomes part of the display of the multimedia document in a temporal context.

Claims

1. A method for displaying a multimedia document comprising the steps of:

retrieving the multimedia document from a first computer system, the multimedia document comprising temporally-dimensioned content;

determining, from an identifier of the multimedia document, that a temporal annotation is associated with the multimedia document;

retrieving the temporal annotation;

displaying the temporally-dimensioned content; and

while displaying the temporally-dimensioned content, displaying the temporal annotation over a range of relative time within the temporally-dimensioned content.

2. The method of claim 1 wherein the identifier of the multimedia document is a globally unique identifier (GUID) that identifies one or more components of the multimedia document, the GUID comprising:

an Internet Protocol (IP) address of a capturing computer that captured the temporally-dimensioned content of the multimedia document; and

a time stamp representing the time the temporally-dimensioned content of the multimedia document was captured.

3. The method of claim 1 wherein the identifier of the multimedia document is a universal resource name within a computer network to which the first computer system is connected.

4. The method of claim 1 wherein the identifier of the multimedia document is a universal resource locator (URL) within a computer network to which the first computer system is connected.

5. The method of claim 1 wherein the identifier of the multimedia document is a realtime transport protocol (RTP) address.

6. The method of claim 1 wherein the step of determining comprises the steps of:

deriving, from the identifier, a location of a database stored in a second computer system; and

determining that the database includes the temporal annotation.

7. The method of claim 6 wherein the first computer system and the second computer system are the same computer system.

8. The method of claim 1 wherein the step of determining comprises the steps of:

retrieving, from a first database, a record which represents an association of the identifier with a location of a second database stored in a second computer system; and

determining that the second database includes the temporal annotation.

9. The method of claim 8 wherein the first computer system and the second computer system are the same computer system.

10. A method for managing temporal annotations of a multimedia document, the multimedia document comprising temporally-dimensioned content, the method comprising the steps of:

receiving a post request which specifies (i) identification data representing an identifier of the multimedia document, (ii) content data representing substantive content of a temporal annotation, and (iii) relative time data representing a range of relative time within the temporally-dimensioned content with which the temporal annotation is associated;

forming a temporal annotation entry which includes the content data and the relative time data; and

storing the temporal annotation entry in a pre-existing temporal annotation database such that the temporal annotation entry is associated with the multimedia document.

11. The method of claim 10 wherein the post request further includes author data which identifies a user as the author of the content data, the method further comprising the step of:

including the author data in the temporal annotation entry.

12. The method of claim 10 further comprising the steps of:

forming temporal annotation identifier data which identifies the temporal annotation entry; and

including the temporal annotation identifier data in the temporal annotation entry.

13. The method of claim 10 further comprising the steps of:

determining a creation time of the temporal annotation entry; and

including creation time stamp data representing the creation time in the temporal annotation entry.

14. The method of claim 13 wherein the step of determining a creation time comprises parsing creation time data representing the creation time from the post request.

15. The method of claim 10 further comprising the steps of:

receiving a retrieve request which specifies the identifier of the multimedia document;

retrieving the temporal annotation entry from the pre-existing temporal annotation database; and

returning, in response to the retrieve request, the content data and the relative time data.

16. A method for selectively incorporating display of a temporal annotation in a display of a multimedia document, the method comprising the steps of:

requesting one or more temporal annotations which are related to the multimedia document from a temporal annotation server;

receiving the one or more temporal annotations from the temporal annotation server;

displaying temporally-dimensioned content of the multimedia document to a user;

filtering the one or more temporal annotations according to filtering criteria selected by the user to identify a selected temporal annotation; and

incorporating display of the selected temporal annotation over a range of relative time within the display of the temporally-dimensioned content.

17. The method of claim 16 wherein the filtering criteria include identifiers of one or more selected authors of the one or more temporal annotations.

18. The method of claim 16 wherein the filtering criteria include specification of a period of time during which the selected temporal annotations are created.

19. The method of claim 18 wherein the period of time is specified as a range of dates.

20. The method of claim 16 wherein the step of filtering comprises the step of:

in the step of requesting, specifying the filtering criteria; and

in the step of receiving, receiving only the selected temporal annotations.

21. A computer readable medium useful in association with a computer which includes a processor and a memory, the computer readable medium including computer instructions which are configured to cause the computer to display a multimedia document by performing the steps of:

retrieving the multimedia document from a first computer system, the multimedia document comprising temporally-dimensioned content;

determining, from an identifier of the multimedia document, that a temporal annotation is associated with the multimedia document;

retrieving the temporal annotation;

displaying the temporally-dimensioned content; and

while displaying the temporally-dimensioned content, displaying the temporal annotation over a range of relative time within the temporally-dimensioned content.

22. The computer readable medium of claim 21 wherein the identifier of the multimedia document is a globally unique identifier (GUID) that identifies one or more components of the multimedia document, the GUID comprising:

an Internet Protocol (IP) address of a capturing computer that captured the temporally-dimensioned content of the multimedia document; and

a time stamp representing the time the temporally-dimensioned content of the multimedia document was captured.

23. The computer readable medium of claim 21 wherein the identifier of the multimedia document is a universal resource name within a computer network to which the first computer system is connected.

24. The computer readable medium of claim 21 wherein the identifier of the multimedia document is a universal resource locator (URL) within a computer network to which the first computer system is connected.

25. The computer readable medium of claim 21 wherein the identifier of the multimedia document is a realtime transport protocol (RTP) address.

26. The computer readable medium of claim 21 wherein the step of determining comprises the steps of:

deriving, from the identifier, a location of a database stored in a second computer system; and

determining that the database includes the temporal annotation.

27. The computer readable medium of claim 26 wherein the first computer system and the second computer system are the same computer system.

28. The computer readable medium of claim 21 wherein the step of determining comprises the steps of:

retrieving, from a first database, a record which represents an association of the identifier with a location of a second database stored in a second computer system; and

determining that the second database includes the temporal annotation.

29. The computer readable medium of claim 28 wherein the first computer system and the second computer system are the same computer system.

30. A computer readable medium useful in association with a computer which includes a processor and a memory, the computer readable medium including computer instructions which are configured to cause the computer to manage temporal annotations of a multimedia document comprising temporally-dimensioned content by performing the steps of:

receiving a post request which specifies (i) identification data representing an identifier of the multimedia document, (ii) content data representing substantive content of a temporal annotation, and (iii) relative time data representing a range of relative time within the temporally-dimensioned content with which the temporal annotation is associated;

forming a temporal annotation entry which includes the content data and the relative time data; and

storing the temporal annotation entry in a pre-existing temporal annotation database such that the temporal annotation entry is associated with the multimedia document.

31. The computer readable medium of claim 30 wherein the post request further includes author data which identifies a user as the author of the content data, and wherein the computer instructions are further configured to cause the computer to perform the additional step of:

including the author data in tile temporal annotation curry.

32. The computer readable medium of claim 30 wherein the computer instructions are further configured to cause the computer to perform the additional steps of:

forming temporal annotation identifier data which identifies the temporal annotation entry; and

including the temporal annotation identifier data in the temporal annotation entry.

33. The computer readable medium of claim 30 wherein the computer instructions are further configured to cause the computer to perform the additional steps of:

determining a creation time of the temporal annotation entry; and

including creation time stamp data representing the creation time in the temporal annotation entry.

34. The computer readable medium of claim 33 wherein the step of determining a creation time comprises the step of:

parsing creation time data representing the creation time from the post request.

35. The computer readable medium of claim 32 wherein the computer instructions are further configured to cause the computer to perform the additional steps of:

receiving a retrieve request which specifies the identifier of the multimedia document;

retrieving the temporal annotation entry from the pre-existing temporal annotation database; and

returning, in response to the retrieve request, the content data and the relative time data.

36. A computer readable medium useful in association with a computer which includes a processor and a memory, the computer readable medium including computer instructions which are configured to cause the computer to selectively incorporate display of a temporal annotation in a display of a multimedia document by performing the steps of:

requesting one or more temporal annotations which are related to the multimedia document from a temporal annotation server;

receiving the one or more temporal annotations from the temporal annotation server;

displaying temporally-dimensioned content of the multimedia document to a user;

filtering the one or more temporal annotations according to filtering criteria selected by the user to identify a selected temporal annotation; and

incorporating display of the selected temporal annotation over a range of relative time within the display of the temporally-dimensioned content.

37. The computer readable medium of claim 36 wherein the filtering criteria include identifiers of one or more selected authors of the one or more temporal annotations.

38. The computer readable medium of claim 36 wherein the filtering criteria include specification of a period of time during which the selected temporal annotations are created.

39. The computer readable medium of claim 38 wherein the period of time is specified as a range of dates.

40. The computer readable medium of claim 36 wherein the step of filtering comprises the steps of:

in the step of requesting, specifying the filtering criteria; and

in the step of receiving, receiving only the selected temporal annotations.

41. A computer system comprising:

a processor;

a memory operatively coupled to the processor; and

a temporal annotation manager which executes in the processor from the memory and which, when executed by the processor, causes the computer to display a multimedia document by performing the steps of;

retrieving the multimedia document from the computer system, the multimedia document comprising temporally-dimensioned content;

determining, from an identifier of the multimedia document, that a temporal annotation is associated with the multimedia document;

retrieving the temporal annotation;

displaying the temporally-dimensioned content; and

while displaying the temporally-dimensioned content, displaying the temporal annotation over a range of relative time within the temporally-dimensioned content.

42. The computer system of claim 41 wherein the identifier of the multimedia document is a globally unique identifier (GUID) that identifies one or more components of the multimedia document, the GUID comprising:

an Internet Protocol (IP) address of a capturing computer that captured the temporally-dimensioned content of the multimedia document; and

a time stamp representing the time the temporally-dimensioned content of the multimedia document was captured.

43. The computer system of claim 41 wherein the identifier of the multimedia document is a universal resource name within a computer network to which the computer system is connected.

44. The computer system of claim 41 wherein the identifier of the multimedia document is a universal resource locator (URL) within a computer network to which the computer system is connected.

45. The computer system of claim 41 wherein the identifier of the multimedia document is a realtime transport protocol (RTP) address.

46. The computer system of claim 41 wherein the step of determining comprises the steps of:

deriving, from the identifier, a location of a database stored in a second computer system; and

determining that the database includes the temporal annotation.

47. The computer system of claim 46 wherein the computer system and the second computer system are the same computer system.

48. The computer system of claim 41 wherein the step of determining comprises the steps of:

retrieving, from a first database, a record which represents an association of the identifier with a location of a second database stored in a second computer system; and

determining that the second database includes the temporal annotation.

49. The computer system of claim 48 wherein the computer system and the second computer system are the same computer system.

50. A computer system comprising:

a processor;

a memory operatively coupled to the processor; and

a temporal annotation manager which executes in the processor from the memory and which, when executed by the processor, causes the computer to manage temporal annotations of a multimedia document comprising temporally-dimensioned content by performing the steps of;

receiving a post request which specifies (i) identification data representing an identifier of the multimedia document, (ii) content data representing substantive content of a temporal annotation, and (iii) relative time data representing a range of relative time within the temporally-dimensioned content with which the temporal annotation is associated;

forming a temporal annotation entry which includes the content data and the relative time data; and

storing the temporal annotation entry in a pre-existing temporal annotation database such that the temporal annotation entry is associated with the multimedia document.

51. The computer system of claim 50 wherein the post request further includes author data which identifies a user as the author of the content data, and wherein the temporal annotation manager causes the computer to perform the additional step of:

including the author data in the temporal annotation entry.

52. The computer system of claim 50 wherein the temporal annotation manager causes the computer to perform the additional steps of:

forming temporal annotation identifier data which identifies the temporal annotation entry; and

including the temporal annotation identifier data in the temporal annotation entry.

53. The computer system of claim 50 wherein the temporal annotation manager causes the computer to perform the additional steps of:

determining a creation time of the temporal annotation entry; and

including creation time stamp data representing the creation time in the temporal annotation entry.

54. The computer system of claim 53 wherein the step of determining a creation time comprises the step of:

parsing creation time data representing the creation time from the post request.

55. The computer system of claim 50 wherein the temporal annotation manager causes the computer to perform the additional steps of:

receiving a retrieve request which specifies the identifier of the multimedia document;

retrieving the temporal annotation entry from the pre-existing temporal annotation database; and

returning, in response to the retrieve request, the content data and the relative time data.

56. A computer system comprising:

a processor;

a memory operatively coupled to the processor; and

a multimedia document player which executes in the processor from the memory and which, when executed by the processor, causes the computer to selectively incorporate display of a temporal annotation in a display of a multimedia document by performing the steps of;

requesting one or more temporal annotations which are related to the multimedia document from a temporal annotation server;

receiving the one or more temporal annotations from the temporal annotation server;

displaying temporally-dimensioned content of the multimedia document to a user;

filtering the one or more temporal annotations according to filtering criteria selected by the user to identify a selected temporal annotation; and

incorporating display of the selected temporal annotation over a range of relative time within the display of the temporally-dimensioned content.

57. The computer system of claim 56 wherein the filtering criteria include identifiers of one or more selected authors of the one or more temporal annotations.

58. The computer system of claim 56 wherein the filtering criteria include specification of a period of time during which the selected temporal annotations are created.

59. The computer system of claim 58 wherein the period of time is specified as a range of dates.

60. The computer system of claim 56 wherein the step of filtering comprises the step of:

in the step of requesting, specifying the filtering criteria; and

in the step of receiving, receiving only the selected temporal annotations.

61. A method of annotating a multimedia document, comprising:

displaying temporally-dimensioned content of the multimedia document; and

synchronizing a temporal annotation to a range of relative time within the temporally-dimensioned content, wherein the synchronizing comprises associating the temporal annotation with the multimedia document, the associating further comprising;

submitting a temporal annotation post request that includes (i) an identifier of the multimedia document, (ii) substantive content of the temporal annotation, and (iii) the range of relative time within the temporally-dimensioned content with which the temporal annotation is associated;

forming a temporal annotation entry that includes the substantive content and the range of relative time; and

storing the temporal annotation entry in a pre-existing temporal annotation database such that the temporal annotation entry is associated with the multimedia document.

62. The method of claim 61, wherein the temporal annotation post request further includes an author identifier specifying the author of the substantive content, the method further comprising:

including the author identifier in the temporal annotation entry.

63. The method of claim 61, further comprising:

forming a temporal annotation identifier to identify the temporal annotation entry; and

including the temporal annotation identifier in the temporal annotation entry.

64. The method of claim 61, further comprising:

determining a creation time of the temporal annotation entry; and

including a creation time stamp representing the creation time in the temporal annotation entry.

65. A method of displaying a multimedia document, comprising:

retrieving the multimedia document from a first computer;

determining from an identifier of the multimedia document that a temporal annotation is associated with the multimedia document;

retrieving the temporal annotation;

displaying temporally-dimensioned content of the multimedia document; and

over a range of relative time within the temporally-dimensioned content, displaying the temporal annotation,

wherein the identifier of the multimedia document is a globally unique identifier (GUID) that identifies one or more components of the multimedia document, the GUID comprising;

an Internet Protocol (IP) address of a capturing computer that captured the temporally-dimensioned content of the multimedia document; and

a time stamp representing the time the temporally-dimensioned content of the multimedia document was captured.

66. A method of recursively annotating a multimedia document, comprising:

displaying temporally-dimensioned content of the multimedia document;

while displaying the temporally-dimensioned content of the multimedia document, displaying a first temporal annotation associated with the multimedia document; and

synchronizing a second temporal annotation to a range of relative time within the first temporal annotation,

wherein the synchronizing comprises associating the second temporal annotation with the multimedia document, the associating further comprising;

submitting a temporal annotation post request that includes (i) an identifier of the multimedia document, (ii) substantive content of the second temporal annotation, and (iii) the range of relative time within the first temporal annotation with which the second temporal annotation is associated;

forming a temporal annotation entry that includes the substantive content and the range of relative time; and

storing the temporal annotation entry in a pre-existing temporal annotation database such that the temporal annotation entry is associated with the multimedia document.

67. The method of claim 66, wherein the temporal annotation post request further includes an author identifier specifying the author of the substantive content, the method further comprising:

including the author identifier in the temporal annotation entry.

68. The method of claim 66, further comprising:

forming a temporal annotation identifier to identify the temporal annotation entry; and

including the temporal annotation identifier in the temporal annotation entry.

69. The method of claim 66, further comprising:

determining a creation time of the temporal annotation entry; and

including a creation time stamp representing the creation time in the temporal annotation entry.

70. A method of displaying a multimedia document, comprising:

retrieving the multimedia document from a first computer, the multimedia document comprising temporally-dimensioned content;

determining from an identifier of the multimedia document that one or more temporal annotations are associated with the multimedia document;

displaying a table of contents associated with the multimedia document, the table of contents comprising the one or more temporal annotations;

based on a selection of a temporal annotation from the table of contents, initiating a display of the temporally-dimensioned content at a relative time within the temporally-dimensioned content, the temporal annotation being synchronized to the relative time.

71. The method of claim 70, further comprising:

retrieving the temporal annotation from a second computer; and

displaying the temporal annotation at the relative time along with the display of the temporally-dimensioned content.

72. The method of claim 70, further comprising:

displaying information about a particular temporal annotation in response to actuating the particular temporal annotation with a user interface device.

73. The method of claim 72, wherein the user interface device is a cursor and the actuating is positioning the cursor over the particular temporal annotation for a predetermined period of time.

74. A method for displaying a multimedia document comprising the steps of:

retrieving the multimedia document from a first computer system, the multimedia document comprising temporally-dimensioned content;

determining, from an identifier of the multimedia document, that one or more temporal annotations are associated with the multimedia document, each of the one or more temporal annotations being associated with a range of relative time within the temporally-dimensioned content;

retrieving the one or more temporal annotations;

displaying the temporally-dimensioned content; and

displaying the one or more temporal annotations.

75. The method of claim 74, wherein the displaying the one or more temporal annotations comprises displaying a table of contents containing selectable items, each selectable item representing a selectable temporal annotation.

76. The method of claim 75, further comprising displaying the temporally-dimensioned content over a range of relative time within the temporally-dimensioned content with which a selected temporal annotation is associated.

77. The method of claim 75, wherein each selectable item comprises an except from a selectable temporal annotation.