Techniques for using paper documents as media templates

• US 7,779,355 B1
• Filed: 03/30/2004
• Issued: 08/17/2010
• Est. Priority Date: 03/30/2004
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey, Hull, Jonathan J., Erol, Berna
• Primary Examiner(s): Hutton; Doug
• Assistant Examiner(s): Tran; Quoc A
• Application Number:
  US10/813,901
• Time in Patent Office: 2,331 Days
• Field of Search: 707/204, 725/1, 382/306, 386/95, 715/512, 715/209, 715/233, 715/704, 715/714, 715/716, 715730-733, 715/273, 715/234, 345/732
• US Class Current: 715/273
• CPC Class Codes: G06F 16/5846 using extracted text

Abstract

Techniques for creating a composite image are provided. The techniques include receiving an electronic representation of a paper document. Features in the electronic representation are then extracted and compared to recorded information to determine matching information. For example, the matching information may be presentations and/or pages in the recorded information. Information is then determined based on the matching information and the received electronic representation. The composite electronic representation is then created using the determined information.

Claims

1. A method in a computer system for creating a composite electronic representation including presentation material information, the method comprising:

scanning a paper document to generate an electronic representation of the document, the document including presentation material having text;

extracting a visual feature from the electronic representation of the document, the visual feature corresponding to at least a portion of the presentation material;

accessing recorded information including audio and visual information recorded during a presentation of the presentation material, applying an optical character recognition (OCR) algorithm to the text to produce recognized text, and comparing the recognized text to the audio of the recorded information to determine a portion of the audio of the recorded information that matches the recognized text, and determining matching information for each matching portion of the recorded information and recognized text using a matching algorithm configured to map the recognized text to a portion of any of a plurality of recorded information that includes audio that matches the recognized text;

generating a user selectable object providing a user with access to the portion of the recorded information determined to match the recognized text, and inserting the user selectable object into the electronic representation of the document when the computer system locates a portion of the recorded information determined to match the recognized text, the computer system thus creating a composite electronic representation of the document including the user selectable object and metadata including the matching information, the user selectable object being placed in a position associated with the extracted feature and allowing the user to access the portion of the recorded information as an embedded video link in the user selectable object by selecting the user selectable object, the user-selectable object being able to access the portion of the recorded information using the metadata in the composite document; and

storing the composite electronic representation as a PDF, HyperText Transfer Language (HTML), Flash or Word formatted document for access by the user or another user accessing the composite electronic document.

2. The method of claim 1, further comprising determining association information for the recorded information that corresponds to the extracted feature.

3. The method of claim 2, wherein the association information comprises time information and source information for recorded information.

4. The method of claim 2, further comprising associating the association information with the determined additional information in the composite electronic representation.

5. The method of claim 4, further comprising:

receiving a selection of the determined additional information in the composite electronic representation; and

using the association information for the determined additional information to access the recorded information.

6. The method of claim 1, further comprising accessing the recorded information using the determined additional information.

7. The method of claim 6, further comprising displaying the accessed recorded information.

8. The method of claim 7, further comprising playing the accessed information.

9. The method of claim 1, further comprising:

performing at least one of emailing, printing, storing, and copying the created composite electronic representation.

10. The method of claim 1, wherein the received electronic representation of the paper document includes notes taken by a user, wherein the created composite electronic representation includes the notes taken by the user.

11. The method of claim 1, further comprising determining a document that includes the recorded information using the extracted feature.

12. The method of claim 11, further comprising determining a portion of the document that includes the information corresponding to the feature.

13. The method of claim 1, wherein the feature comprises an identifier to a location in the recorded information, wherein the information in the recorded information corresponding to the feature is determined using the identifier.

14. The method of claim 13, wherein the identifier comprises at least one of a barcode and signature information.

15. The method of claim 1, wherein receiving the electronic representation comprises receiving a scan of the document, the document being a paper document.

16. The method of claim 1, wherein receiving the electronic representation comprises determining an electronic image of the document, the document being a paper document.

17. The method of claim 1, wherein receiving the electronic representation comprises receiving the electronic representation in response to an input from a user indicating that the composite electronic representation should be created.

18. The method of claim 1, wherein the document comprises a paper document.

19. A method in a computer system for creating a composite electronic representation of a document using information, having audio content, recorded during a presentation, the method comprising:

scanning a paper document having text to generate an electronic representation of the document for the presentation;

applying an optical character recognition (OCR) algorithm to the text from the electronic representation to generate OCR determined text;

accessing recorded information including audio and visual information recorded during the presentation, and using a matching algorithm to compare the OCR determined text to the audio of the recorded information to determine a portion of the recorded information that matches the OCR determined text, the matching algorithm configured to map the OCR determined text to a portion of any of a plurality of recorded information having audio that matches and generate matching information in response thereto;

generating composite information based on the portion of the recorded information having audio that corresponds to the OCR determined text in the electronic representation of the document;

generating a user selectable object providing a user with access to the portion of the recorded information corresponding to the OCR determined text, and inserting the user selectable object and metadata including the matching information into the electronic representation of the document when the computer system locates a portion of the recorded information corresponding to the OCR determined text, the computer system thus creating a composite electronic representation of the document including the user selectable object and the metadata, the user selectable object being placed in a position associated with the OCR determined text and allowing the user to access the composite information in an application displaying the composite electronic representation or a separate application by selecting the user selectable object, the user-selectable object being able to access the portion of the recorded information using an embedded video link in the user selectable object; and

storing the composite electronic representation as a PDF, HyperText Transfer Language (HTML), Flash or Word formatted document for access by the user or another user accessing the composite electronic document.

20. The method of claim 19, further comprising determining association information for the recorded information that corresponds to the OCR determined text.

21. The method of claim 20, wherein the association information comprises time information for the recorded information, the time information indicating a time when information related to the OCR determined text was presented during the presentation.

22. The method of claim 21, further comprising:

using the association information for the determined information in the composite electronic representation to access recorded information for the presentation at a time indicated by the time information.

23. The method of claim 19, further comprising accessing recorded information using the determined composite information.

24. The method of claim 23, further comprising displaying the accessed recorded information.

25. The method of claim 24, further comprising playing the accessed information.

26. The method of claim 19, further comprising performing at least one of emailing, printing, storing, and copying the created composite electronic representation.

27. The method of claim 19, wherein the document comprises a paper document.

28. The method of claim 19, wherein the OCR determined text comprises an identifier to a location in the recorded information, wherein the information in the recorded information that corresponds to the OCR determined text is determined using the identifier.

29. The method of claim 28, wherein the identifier comprises at least one of a barcode and signature information.

30. The method of claim 19, wherein receiving the electronic representation comprises receiving the electronic representation in response to an input from a user indicating that the composite electronic representation should be created.

31. A computer program product stored on a computer-readable medium for creating a composite electronic representation, the computer program product comprising:

code for scanning a paper document having text to generate an electronic representation of the document, the document including presentation material;

code for applying an optical character recognition (OCR) algorithm to the text from the electronic representation of the document to generate OCR determined text, the OCR determined text corresponding to at least a portion of the presentation material;

code for accessing recorded information including of audio and visual information recorded during a presentation of the presentation material, and comparing the OCR determined text to the recorded information to determine a portion of the audio of the recorded information that matches the OCR determined text, and determining matching information for each matching portion of the audio of the recorded information and the OCR determined text using a matching algorithm configured to map the OCR determined text to a portion of any of a plurality of recorded information having audio that matches the OCR determined text;

code for generating a user selectable object providing a user with access to the portion of the recorded information determined to match the OCR determined text, and inserting the user selectable object and metadata including the matching information into the electronic representation of the document when the computer system locates a portion of the audio of the recorded information corresponding to the OCR determined text, the computer system thus creating a composite electronic representation of the document including the user selectable object, the user selectable object being placed in a position associated with the OCR determined text and allowing the user to access the portion of the recorded information in an application displaying the composite electronic representation or a separate application by selecting the user selectable object, the user-selectable object being able to access the portion of the recorded information using an embedded video link in the user selectable object; and

code for storing the composite electronic representation as a PDF, HyperText Transfer Language (HTML), Flash or Word formatted document for access by the user or another user accessing the composite electronic document.

32. The computer program product of claim 31, further comprising code for determining association information for the recorded information that corresponds to the OCR determined text.

33. The computer program product of claim 32, further comprising code for associating the association information with the determined additional information in the composite electronic representation.

34. The computer program product of claim 33, further comprising:

code for receiving a selection of the determined additional information in the composite electronic representation; and

code for using the association information for the determined additional information to access the recorded information.

35. The computer program product of claim 31, further comprising code for accessing the recorded information using the determined additional information.

36. The computer program product of claim 31, further comprising:

code for performing at least one of emailing, printing, storing, displaying, playing, and copying the created composite electronic representation.

37. The computer program product of claim 31, wherein the received electronic representation of the document includes notes taken by a user, wherein the created composite electronic representation includes the notes taken by the user.

38. The computer program product of claim 31, wherein the document comprises a paper document.

39. A computer program product stored on a computer-readable medium for creating a composite electronic representation of a document having text using information recorded during a presentation, the computer program product comprising:

code for scanning a paper document to generate an electronic representation of the document for the presentation, the electronic representation including text;

code for applying an optical character recognition (OCR) algorithm to the text from the electronic representation to generate OCR determined text;

code for accessing recorded information including audio and visual information recorded during the presentation, and using a matching algorithm to compare the OCR determined text to the audio of the recorded information to determine a portion of the recorded information that matches the OCR determined text, the matching algorithm configured to map the OCR determined text to a portion of any of a plurality of recorded information with audio that matches the OCR determined text and generate matching information in response thereto;

code for generating composite information based on the portion of the recorded information that corresponds to the OCR determined text and the electronic representation of the document;

code for generating a user selectable object providing a user with access to the portion of the audio of the recorded information corresponding to the OCR determined text, and inserting the user selectable object and metadata including the matching information into the electronic representation of the document when the computer system locates a portion of the recorded information corresponding to the OCR determined text, the computer system thus creating a composite electronic representation of the document including the user selectable object and the metadata, the user selectable object being placed in a position associated with the text and allowing the user to access the composite information in an application displaying the composite electronic representation or a separate application by selecting the user selectable object, the user-selectable object being able to access the portion of the recorded information using an embedded video link in the user selectable object; and

code for storing the composite electronic representation as a PDF, HyperText Transfer Language (HTML), Flash or Word formatted document for access by the user or another user accessing the composite electronic document.

40. The computer program product of claim 39, further comprising code for determining association information for the recorded information that corresponds to the text.

41. The computer program product of claim 40, wherein the association information comprises time information for recorded information, the time information indicating a time when information related to the text was presented during the presentation.

42. The computer program product of claim 41, further comprising:

code for using the association information for the determined composite information in the composite electronic representation to access recorded information for the presentation at a time indicated by the time information.

43. The computer program product of claim 39, further comprising code for performing at least one of emailing, printing, storing, displaying, playing, and copying the created representation.

44. The computer program product of claim 39, wherein the document comprises a paper document.

45. A data processing system for creating a composite electronic representation, the data processing system comprising:

a processor;

a memory coupled to the processor, the memory configured to store a plurality of modules for execution by the processor, the plurality of modules comprising;

logic to scan a paper document to generate an electronic representation of the document, the document including text and presentation material;

logic to apply a optical character recognition (OCR) algorithm to the text from the electronic representation of the document to generate OCR determined text, the OCR determined feature corresponding to at least a portion of the text and presentation material;

logic to access recorded information including audio and visual information recorded during a presentation of the presentation material, and compare the OCR determined text to the recorded information to determine an audio portion of the recorded information that matches the OCR determined text, and determining matching information for each matching portion of the recorded information and OCR determined text using a matching algorithm configured to map the OCR determined text to a portion of any of a plurality of recorded information having audio that matches the OCR determined text;

logic to generate a user selectable object providing a user with access to the portion of the recorded information corresponding to determined to match the OCR determined text, and inserting the user selectable object and metadata including the matching information into the electronic representation of the document when the computer system locates a portion of the recorded information corresponding to the OCR determined text, the computer system thus creating a composite electronic representation of the document including the user selectable object, the user selectable object being placed in a position associated with the OCR determined text and allowing the user to access the portion of the recorded information in an application displaying the composite electronic representation or a separate application by selecting the user selectable object, the user-selectable object being able to access the portion of the recorded information using an embedded video link in the user selectable object; and

logic to store the composite electronic representation as a PDF, HyperText Transfer Language (HTML), Flash or Word formatted document for access by the user or another user accessing the composite electronic document.

46. The data processing system of claim 45, further comprising logic to determine association information for the recorded information that corresponds to the text extracted feature.

47. The data processing system of claim 46, further comprising logic to associate the association information with the determined additional information in the composite electronic representation.

48. The data processing system of claim 47, further comprising:

logic to receive a selection of the determined additional information in the composite electronic representation; and

logic to use the association information for the determined information to access the recorded information.

49. The data processing system of claim 45, further comprising logic to access recorded information using the determined additional information.

50. The data processing system of claim 45, further comprising:

logic to perform at least one of emailing, printing, storing, displaying, playing, and copying the created composite electronic representation.

51. The data processing system of claim 45, wherein the received electronic representation includes notes taken by a user, wherein the created composite electronic representation includes the notes taken by the user.

52. The data processing system of claim 45, wherein the document comprises a paper document.

53. A data processing system creating a composite electronic representation of a document using information recorded during a presentation, the data processing system comprising:

a processor;

a memory coupled to the processor, the memory configured to store a plurality of modules for execution by the processor, the plurality of modules comprising;

logic to scan a paper document to generate an electronic representation of the document for the presentation having text;

logic to apply an optical character recognition (OCR) algorithm to the electronic representation to generate OCR determined text;

logic to access recorded information including audio and visual information recorded during the presentation, and using a matching algorithm to compare the OCR determined text to the audio of the recorded information to determine a portion of the recorded information that matches the OCR determined text, the matching algorithm configured to map the OCR determined text to a portion of any of a plurality of recorded information having audio that matches the OCR determined text and generate matching information in response thereto;

logic to generate composite information based on the audio of the portion of the recorded information that corresponds to the OCR determined text in the electronic representation of the document;

logic to generate a user selectable object providing a user with access to the portion of the recorded information corresponding to the OCR determined text in, and inserting the user selectable object and metadata including the matching information into the electronic representation of the document when the computer system locates a portion of the recorded information corresponding to the OCR determined text in, the computer system thus creating a composite electronic representation of the document including the user selectable object and the metadata, the user selectable object being placed in a position associated with the extracted feature and allowing the user to access the composite information in an application displaying the composite electronic representation or a separate application by selecting the user selectable object, the user-selectable object being able to access the portion of the recorded information using an embedded video link in the user selectable object; and

logic to store the composite electronic representation as a PDF, HyperText Transfer Language (HTML), Flash or Word formatted document for access by the user or another user accessing the composite electronic document.

54. The data processing system of claim 53, further comprising logic to determine association information for the recorded information that corresponds to the feature.

55. The data processing system of claim 54, wherein the association information comprises time information for recorded information, the time information indicating a time when information related to the feature was presented during the presentation.

56. The data processing system of claim 55, further comprising:

logic to use the association information for the determined composite information in the composite electronic representation to access the recorded information for the presentation at a time indicated by the time information.

57. The data processing system of claim 53, further comprising logic to perform at least one of emailing, printing, storing, displaying, playing, and copying the created representation.

58. The data processing system of claim 53, wherein the document comprises a paper document.