Techniques for annotating multimedia information

• US 7,263,671 B2
• Filed: 11/19/2001
• Issued: 08/28/2007
• Est. Priority Date: 09/09/1998
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey, Hull, Jonathan J.
• Primary Examiner(s): BAUTISTA, XIOMARA L
• Application Number:
  US10/001,849
• Publication Number:
  US 20050216851A1
• Time in Patent Office: 2,108 Days
• Field of Search: 345/700, 345/716, 345/717, 345/719, 345/727, 345/764, 345/835, 345/838, 345/839, 345/855, 358/1.1, 358/1.15, 358/1.18, 358/530, 358/540, 705/55, 715/479, 715/500, 715/500.1, 715/501.1, 715/513, 715/526, 715/527, 715/700, 715/716, 715/717, 715/719, 715/727, 715/764, 715/835, 715/838, 715/839, 715/855
• US Class Current: 715/839
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/186 Templates

Abstract

The present invention provides techniques for generating a printable representation of multimedia information that can be printed on a paper medium to provide a paper-based interface for the multimedia information. The printable representation for the multimedia information is annotated to identify locations of information in the multimedia information that may be of interest to a user. A multimedia paper document generated by printing the annotated printable representation on a paper medium displays the annotations. The annotations provide visual indications of information relevant to the user.

Claims

1. A computer-implemented method of generating a paper document for a electronically stored multimedia document storing multimedia information, the multimedia information including video information, the method comprising:

accepting user input identifying a first concept of interest;

analyzing the multimedia information stored by the multimedia document to identify, absent direct human interaction, information relevant to the first concept of interest; and

printing the multimedia information on a paper medium to generate the paper document comprising one or more printed pages, wherein information that is identified to be relevant to the first concept of interest is annotated when printed on the one or more pages.

2. The method of claim 1 wherein the user input identifying the first concept of interest is stored in a user profile.

3. The method of claim 1 further comprising:

printing a relevance indicator on at least one page of the one or more pages, the relevance indicator indicating a degree of relevance of the multimedia information stored by the multimedia document to the first concept of interest.

4. The method of claim 1 further comprising:

accepting user input identifying a second concept of interest;

analyzing the multimedia information stored by the multimedia document to identify information relevant to the second concept of interest; and

printing the multimedia information on the one or more printed pages such that the information that is identified to be relevant to the second concept of interest is annotated when printed on the one or more pages.

5. The method of claim 4 wherein the information printed on the one or more pages that is identified to be relevant to the first topic of interest by analyzing the multimedia information is annotated using a first style and the information printed on the one or more pages that is identified to be relevant to the second topic of interest by analyzing the multimedia information is annotated using a second style different from the first style.

6. The method of claim 1 wherein the multimedia information comprises audio information, and wherein:

analyzing the multimedia information comprises;

generating a text transcript for the audio information; and

analyzing the text transcript to identify information in the text transcript that is relevant to the first concept of interest; and

printing the multimedia information on the paper medium comprises;

printing the text transcript on the one or more pages such that the information in the text transcript identified to be relevant to the first concept of interest is annotated when printed on the one or more pages.

7. The method of claim 6 wherein:

accepting user input identifying the first concept of interest comprises receiving information identifying a plurality of words associated with the first concept of interest; and

analyzing the text transcript comprises identifying words in the text transcript that match words in the plurality of words associated with the first concept of interest; and

printing the text transcript on the one or more pages comprises annotating the one or more words in the text transcript printed on the one or more pages that matched words in the plurality of words associated with the first concept of interest.

8. The method of claim 1 wherein the multimedia information comprises closed-caption text information, and wherein:

analyzing the multimedia information comprises analyzing the closed-caption text information to identify text information that is relevant to the first concept of interest; and

printing the multimedia information on the paper medium comprises printing the closed-caption text on the one or more pages such that the text information printed on the one or more pages that is identified to be relevant to the first concept of interest is annotated.

9. The method of claim 8 wherein:

accepting user input identifying the first concept of interest comprises receiving information identifying a plurality of words associated with the first concept of interest;

analyzing the closed-caption text information comprises identifying words in the closed-caption text information that match words in the plurality of words associated with the first concept of interest; and

printing the closed-caption text information on the one or more pages comprises annotating the one or more words in the closed-caption text information printed on the one or more pages that matched words in the plurality of words associated with the first concept of interest.

10. The method of claim 1 wherein:

accepting user input identifying the first concept of interest comprises receiving information identifying video features associated with the first concept of interest;

analyzing the multimedia information comprises;

extracting a plurality of video frames from the video information; and

analyzing the plurality of video frames to identify one or more video frames that comprise at least one video feature from the video features associated with the first concept of interest; and

printing the multimedia information on the paper medium comprises printing the one or more video frames on the one or more pages with annotations.

11. The method of claim 1 wherein:

analyzing the multimedia information stored by the multimedia document to identify information relevant to the first concept of interest comprises;

generating a printable representation for the multimedia information stored by the multimedia document, the printable representation capable of being printed on a paper medium;

searching the printable representation to identify one or more sections of the printable representation that comprise information relevant to the first concept of interest; and

annotating the one or more sections of the printable representation identified to comprise information relevant to the first concept of interest; and

printing the multimedia information on a paper medium to generate the one or more printed pages comprises printing the printable representation on the one or more pages such that the one or more sections of the printable representation identified to comprise information relevant to the first concept of interest are annotated when printed on the one or more pages.

12. A system for generating a paper document for a electronically stored multimedia document storing multimedia information, the multimedia information including video information, the system comprising:

a processor; and

a memory coupled to the processor, the memory configured to store a plurality of code modules for execution by the processor, the plurality of code modules comprising;

a code module for accepting user input identifying a first concept of interest;

a code module for analyzing the multimedia information stored by the multimedia document to identify, absent direct human interaction, information relevant to the first concept of interest; and

a code module for printing the multimedia information on a paper medium to generate one or more printed pages such that the information that is identified to be relevant to the first concept of interest is annotated when printed on the one or more pages, the one or more pages comprising one or more pages.

13. The system of claim 12 wherein the user input identifying the first concept of interest is stored in a user profile.

14. The system of claim 12, wherein the plurality of code modules further comprises:

a code module for printing a relevance indicator on at least one page from the one or more pages, the relevance indicator indicating a degree of relevance of the multimedia information stored by the multimedia document to the first concept of interest.

15. The system of claim 12, wherein the plurality of code modules further comprises:

a code module for accepting user input identifying a second concept of interest;

a code module for analyzing the multimedia information stored by the multimedia document to identify information relevant to the second concept of interest; and

a code module for printing the multimedia information on the one or more printed pages such that the information that is identified to be relevant to the second concept of interest is annotated when printed on the one or more pages.

16. The system of claim 15 wherein the information printed on the one or more pages that is identified to be relevant to the first topic of interest by analyzing the multimedia information is annotated using a first style and the information printed on the one or more pages that is identified to be relevant to the second topic of interest by analyzing the multimedia information is annotated using a second style different from the first style.

17. The system of claim 12 wherein the multimedia information comprises audio information, and wherein:

the code module for analyzing the multimedia information comprises;

a code module for generating a text transcript for the audio information; and

a code module for analyzing the text transcript to identify information in the text transcript that is relevant to the first concept of interest; and

the code module for printing the multimedia information on the paper medium comprises;

a code module for printing the text transcript on the one or more pages such that the information in the text transcript identified to be relevant to the first concept of interest is annotated when printed on the one or more pages.

18. The system of claim 17 wherein:

the code module for accepting user input identifying the first concept of interest comprises a code module for receiving information identifying a plurality of words associated with the first concept of interest; and

the code module for analyzing the text transcript comprises a code module for identifying words in the text transcript that match words in the plurality of words associated with the first concept of interest; and

the code module for printing the text transcript on the one or more pages comprises a code module for annotating the one or more words in the text transcript printed on the one or more pages that matched words in the plurality of words associated with the first concept of interest.

19. The system of claim 12 wherein the multimedia information comprises closed-caption text information, and wherein:

the code module for analyzing the multimedia information comprises a code module for analyzing the closed-caption text information to identify text information that is relevant to the first concept of interest; and

the code module for printing the multimedia information on the paper medium comprises a code module for printing the closed-caption text on the one or more pages such that the text information printed on the one or more pages that is identified to be relevant to the first concept of interest is annotated.

20. The system of claim 19 wherein:

the code module for accepting user input identifying the first concept of interest comprises a code module for receiving information identifying a plurality of words associated with the first concept of interest;

the code module for analyzing the closed-caption text information comprises a code module for identifying words in the closed-caption text information that match words in the plurality of words associated with the first concept of interest; and

the code module for printing the closed-caption text information on the one or more pages comprises a code module for annotating the one or more words in the closed-caption text information printed on the one or more pages that matched words in the plurality of words associated with the first concept of interest.

21. The system of claim 12 wherein:

the code module for accepting user input identifying the first concept of interest comprises a code module for receiving information identifying video features associated with the first concept of interest;

the code module for analyzing the multimedia information comprises;

a code module for extracting a plurality of video frames from the video information; and

a code module for analyzing the plurality of video frames to identify one or more video frames that comprise at least one video feature from the video features associated with the first concept of interest; and

the code module for printing the multimedia information on the paper medium comprises a code module for printing the one or more video frames on the one or more pages with annotations.

22. The system of claim 12 wherein:

the code module for analyzing the multimedia information stored by the multimedia document to identify information relevant to the first concept of interest comprises;

a code module for generating a printable representation for the multimedia information stored by the multimedia document, the printable representation capable of being printed on a paper medium;

a code module for searching the printable representation to identify one or more sections of the printable representation that comprise information relevant to the first concept of interest; and

a code module for annotating the one or more sections of the printable representation identified to comprise information relevant to the first concept of interest; and

the code module for printing the multimedia information on a paper medium to generate the one or more printed pages comprises a code module for printing the printable representation on the one or more pages such that the one or more sections of the printable representation identified to comprise information relevant to the first concept of interest are annotated when printed on the one or more pages.

23. A computer program product stored on a computer-readable storage medium for generating a paper document for a electronically stored multimedia document storing multimedia information, the multimedia information including video information, the computer program product comprising:

code for accepting user input identifying a first concept of interest;

code for analyzing the multimedia information stored by the multimedia document to identify, absent direct human interaction, information relevant to the first concept of interest; and

code for printing the multimedia information on a paper medium to generate the paper document comprising one or more printed pages such that the information that is identified to be relevant to the first concept of interest is annotated when printed on the one or more pages, the one or more pages comprising one or more pages.

24. The computer program product of claim 23 wherein the multimedia information comprises audio information, and wherein:

the code for analyzing the multimedia information comprises;

code for generating a text transcript for the audio information; and

code for analyzing the text transcript to identify information in the text transcript that is relevant to the first concept of interest; and

the code for printing the multimedia information on the paper medium comprises;

code for printing the text transcript on the one or more pages such that the information in the text transcript identified to be relevant to the first concept of interest is annotated when printed on the one or more pages.

25. The computer program product of claim 23 wherein the multimedia information comprises closed-caption text information, and wherein:

the code for analyzing the multimedia information comprises code for analyzing the closed-caption text information to identify text information that is relevant to the first concept of interest; and

the code for printing the multimedia information on the paper medium comprises code for printing the closed-caption text on the one or more pages such that the text information printed on the one or more pages that is identified to be relevant to the first concept of interest is annotated.

26. The computer program product of claim 23 wherein:

the code for accepting user input identifying the first concept of interest comprises receiving information identifying video features associated with the first concept of interest;

the code for analyzing the multimedia information comprises;

code for extracting a plurality of video frames from the video information; and

code for analyzing the plurality of video frames to identify one or more video frames that comprise at least one video feature from the video features associated with the first concept of interest; and

the code for printing the multimedia information on the paper medium comprises code for printing the one or more video frames on the one or more pages with annotations.

27. The computer program product of claim 23 wherein:

the code for analyzing the multimedia information stored by the multimedia document to identify information relevant to the first concept of interest comprises;

code for generating a printable representation for the multimedia information stored by the multimedia document, the printable representation capable of being printed on a paper medium;

code for searching the printable representation to identify one or more sections of the printable representation that comprise information relevant to the first concept of interest; and

code for annotating the one or more sections of the printable representation identified to comprise information relevant to the first concept of interest; and

the code for printing the multimedia information on a paper medium to generate the one or more printed pages comprises code for printing the printable representation on the one or more pages such that the one or more sections of the printable representation identified to comprise information relevant to the first concept of interest are annotated when printed on the one or more pages.