Generating augmented notes and synchronizing notes and document portions based on timing information

• US 7,415,667 B2
• Filed: 01/31/2003
• Issued: 08/19/2008
• Est. Priority Date: 01/31/2003
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Rhodes, Bradley J.
• Primary Examiner(s): Paula; Cesar B
• Application Number:
  US10/355,378
• Publication Number:
  US 20040153969A1
• Time in Patent Office: 2,027 Days
• Field of Search: 715/500, 715/512, 715/541, 715/200, 715/263, 715/243, 715255-256, 715/262, 715/268, 382175-179, 382/187, 382/173, 382/184, 382188-190, 345467-468
• US Class Current: 715/263
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

Techniques for generating a paper document that integrates notes written by a user and different types of captured or recorded information. A user can take written notes during an information communication event. While taking the notes, the user can identify, from information captured during the information communication event, information that is of interest to the user and that the user would like to integrate with or incorporate into the user'"'"'s notes. A paper document is generated by printing a visual representation of the notes written by the user and the visual representations of portions of information that a user would like to integrate with the user'"'"'s notes.

Claims

1. A method of generating a paper document based upon information written by a user on one or more sheets of paper and based upon information captured during an information communication event by one or more information capture devices, the method comprising:

accessing first information representative of markings left by the user on the one or more sheets of paper, the markings representative of information written by the user on the one or more sheets of paper;

determining, from the information captured by the one or more information capture devices, one or more portions of information;

wherein determining the one or more portions of information comprises;

identifying one or more marks from the first information, each mark indicating that a portion of information from the information captured by the one or more information capture devices is to be integrated with the information written by the user;

determining time information associated with each mark in the one or more marks; and

identifying, from the information captured by the one or more information capture devices, a portion of information for each mark in the one or more marks based upon the time information associated with the mark and based upon time information associated with the information captured by the one or more information capture devices;

generating a visual representation for each portion of information in the one or more portions of information; and

generating a paper document wherein contents of the paper document comprise a visual representation of the markings representative of information written by the user on the one or more sheets of paper and the visual representations generated for the one or more portions of information.

2. The method of claim 1 wherein identifying the one or more marks from the first information comprises:

accessing template information identifying locations of one or more special areas on the one or more sheets of paper used by the user to write the information; and

identifying, from the first information, one or more marks that are located on the one or more special areas identified by the template information.

3. The method of claim 1 wherein a special character written by the user is identified as a mark.

4. The method of claim 1 wherein identifying the portion of information for each mark in the one or more marks comprises:

identifying an information capture device from the one or more information capture devices for the mark; and

selecting a portion of information captured by the information capture device identified for the mark.

5. The method of claim 4 wherein identifying the information capture device for the mark comprises:

determining a type of information corresponding to the mark;

identifying a set of information capture devices from the one or more information capture devices that are configured to record the type of information corresponding to the mark; and

selecting an information capture device from the set of information capture devices for the mark.

6. The method of claim 1 wherein generating the paper document comprises:

generating an electronic representation comprising a visual representation of the information written by the user and the visual representations generated for the one or more portions of information; and

printing the electronic representation on one or more paper pages to generate the paper document.

7. The method of claim 6 wherein generating the electronic representation comprises:

determining a location in the visual representation of the information written by the user for placing each visual representation; and

generating the electronic representation such that each visual representation is placed in the visual representation of the information in the location determined for the visual representation.

8. The method of claim 1 wherein generating the visual representation for each portion of information in the one or more portions of information comprises:

determining a type of information for each portion of information; and

generating the visual representation for a portion of information based upon the type of information determined for the portion of information.

9. The method of claim 1 wherein the one or more portions of information include a first portion comprising audio information and wherein generating the visual representation comprises transcribing the audio information in the first portion.

10. The method of claim 1 wherein the one or more portions of information include a first portion comprising video information and wherein generating the visual representation comprises selecting a keyframe from the video information of the first portion.

11. The method of claim 1 wherein determining the one or more portions of information comprises:

identifying a signal that caused capture of information by a first information capture device from the one or more information capture devices; and

identifying information captured by the first information capture device in response to the signal.

12. The method of claim 11 wherein the contents of the paper document comprise a visual representation of the information captured by the first information capture device in response to the signal.

13. The method of claim 1 wherein the one or more portions of information includes a first portion and determining the one or more portions of information comprises selecting the first portion when a threshold associated with a rule has been satisfied.

14. A method of generating a paper document based upon information written by a user on a sheet of paper and based upon information captured by an information capture device, the method comprising:

accessing first information representative of markings left by the user on the sheet of paper, the markings representative of information written by the user on the sheet of paper;

determining a portion of information from information captured by the information capture device;

wherein determining the portion of information comprises;

identifying, from the first information, a mark written by the user determining time information associated with the mark; and

identifying the portion of information based upon the time information associated with the mark and based upon time information associated with the information captured by the information capture device;

generating a visual representation for the portion of information; and

printing a visual representation of the information written by the user and the visual representation generated for the portion of information on one or more paper pages to generate the paper document.

15. The method of claim 14 wherein determining the portion of information comprises:

identifying a first signal; and

identifying, from the information captured by the information capture device, a portion of information captured by the information capture device in response to the signal.

16. The method of claim 14 wherein determining the portion of information comprises selecting the portion of information when a threshold associated with a rule has been satisfied.

17. A system for generating a paper document based upon information written by a user on one or more sheets of paper and based upon information captured during an information communication event by one or more information capture devices, the system comprising:

a communication module configured to receive first information representative of markings left by the user on the one or more sheets of paper, the markings representative of information written by the user on the one or more sheets of paper;

a processor; and

a memory coupled to the processor, the memory configured to store a plurality of code modules for execution by the processor, the plurality of code module comprising;

a code module for determining, from the information captured by the one or more information capture devices, one or more portions of information;

wherein the code module for determining the one or more portions of information comprises;

a code module for identifying one or more marks from the first information, each mark indicating that a portion of information from the information captured by the one or more information capture devices is to be integrated with the information written by the user;

a code module for determining time information associated with each mark in the one or more marks; and

a code module for identifying, from the information captured by the one or more information capture devices, a portion of information for each mark in the one or more marks based upon the time information associated with the mark and based upon time information associated with the information captured by the one or more information capture devices;

a code module for generating a visual representation for each portion of information in the one or more portions of information; and

a code module for generating a paper document wherein contents of the paper document comprise a visual representation of the information written by the user on the one or more sheets of paper and the visual representations generated for the one or more portions of information.

18. The system of claim 17 wherein the code module for identifying the one or more marks from the first information comprises:

a code module for accessing template information identifying locations of one or more special areas on the one or more sheets of paper used by the user to write the information; and

a code module for identifying, from the first information, one or more marks that are located on the one or more special areas identified by the template information.

19. The system of claim 17 wherein a special character written by the user is identified as a mark.

20. The system of claim 17 wherein the code module for identifying the portion of information for each mark in the one or more marks comprises:

a code module for identifying an information capture device from the one or more information capture devices for the mark; and

a code module for selecting a portion of information captured by the information capture device identified for the mark.

21. The system of claim 20 wherein the code module for identifying the information capture device for the mark comprises:

a code module for determining a type of information corresponding to the mark;

a code module for identifying a set of information capture devices from the one or more information capture devices that are configured to record the type of information corresponding to the mark; and

a code module for selecting an information capture device from the set of information capture devices for the mark.

22. The system of claim 17 wherein the code module for generating the paper document comprises:

a code module for generating an electronic representation comprising a visual representation of the information written by the user and the visual representations generated for the one or more portions of information; and

a code module for printing the electronic representation on one or more paper pages to generate the paper document.

23. The system of claim 22 wherein the code module for generating the electronic representation comprises:

a code module for determining a location in the visual representation of the information written by the user for placing each visual representation; and

a code module for generating the electronic representation such that each visual representation is placed in the visual representation of the information in the location determined for the visual representation.

24. The system of claim 17 wherein the code module for generating the visual representation for each portion of information in the one or more portions of information comprises:

a code module for determining a type of information for each portion of information; and

a code module for generating the visual representation for a portion of information based upon the type of information determined for the portion of information.

25. The system of claim 17 wherein the one or more portions of information include a first portion comprising audio information and wherein the code module for generating the visual representation comprises a code module for transcribing the audio information in the first portion.

26. The system of claim 17 wherein the one or more portions of information include a first portion comprising video information and wherein the code module for generating the visual representation comprises a code module for selecting a keyframe from the video information of the first portion.

27. The system of claim 17 wherein the code module for determining the one or more portions of information comprises:

a code module for identifying a signal that caused capture of information by a first information capture device from the one or more information capture devices; and

a code module for identifying information captured by the first information capture device in response to the signal.

28. The system of claim 27 wherein the contents of the paper document comprise a visual representation of the information captured by the first information capture device in response to the signal.

29. The system of claim 17 wherein the one or more portions of information includes a first portion and the code module for determining the one or more portions of information comprises a code module for selecting the first portion when a threshold associated with a rule has been satisfied.

30. A system for generating a paper document based upon information written by a user on a sheet of paper and based upon information captured by an information capture device, the system comprising:

a communication module configured to receive first information representative of markings left by the user on the sheet of paper, the markings representative of information written by the user on the sheet of paper;

a processor; and

a memory coupled to the processor, the memory configured to store a plurality of code modules for execution by the processor, the plurality of code module comprising;

a code module for determining a portion of information from information captured by the information capture device;

wherein the code module for determining the portion of information comprises;

a code module for identifying, from the first information, a mark written by the user;

a code module for determining time information associated with the mark; and

a code module for identifying the portion of information based upon the time information associated with the mark and based upon time information associated with the information captured by the information capture device;

a code module for generating a visual representation for the portion of information; and

a code module for printing a visual representation of the information written by the user and the visual representation generated for the portion of information on one or more paper pages to generate the paper document.

31. The system of claim 30 wherein the code module for determining the portion of information comprises:

a code module for identifying a first signal; and

a code module for identifying, from the information captured by the information capture device, a portion of information captured by the information capture device in response to the signal.

32. The system of claim 30 wherein the code module for determining the portion of information comprises selecting the portion of information when a threshold associated with a rule has been satisfied.

33. A computer program, stored on computer readable medium, for generating a paper document based upon information written by a user on a sheet of paper and based upon information captured by an information capture device, the computer program comprising instructions for:

accessing first information representative of markings left by the user on the sheet of paper, the markings representative of information written by the user on the sheet of paper;

determining a portion of information from information captured by the information capture device;

wherein the instructions for determining the portion of information comprise instructions for;

identifying, from the first information, a mark written by the user;

determining time information associated with the mark; and

identifying the portion of information based upon the time information associated with the mark and based upon time information associated with the information captured by the information capture device;

generating a visual representation for the portion of information; and

printing a visual representation of the information written by the user and the visual representation generated for the portion of information on one or more paper pages to generate the paper document.

34. The computer program of claim 33 wherein the instructions for determining the portion of information comprise instructions for:

identifying a first signal; and

identifying, from the information captured by the information capture device, a portion of information captured by the information capture device in response to the signal.

35. The computer program of claim 33 wherein the instructions for determining the portion of information comprise instructions for selecting the portion of information when a threshold associated with a rule has been satisfied.

36. A computer program, stored on computer readable medium, for generating a paper document based upon information written by a user on one or more sheets of paper and based upon information captured during an information communication event by one or more information capture devices, the computer program comprising instructions for:

accessing first information representative of markings left by the user on the one or more sheets of paper, the markings representative of information written by the user on the one or more sheets of paper;

determining, from the information captured by the one or more information capture devices, one or more portions of information;

wherein the instructions for determining the one or more portions of information comprise instructions for;

identifying one or more marks from the first information, each mark indicating that a portion of information from the information captured by the one or more information capture devices is to be integrated with the information written by the user;

determining time information associated with each mark in the one or more marks; and

identifying, from the information captured by the one or more information capture devices, a portion of information for each mark in the one or more marks based upon the time information associated with the mark and based upon time information associated with the information captured by the one or more information capture devices;

generating a visual representation for each portion of information in the one or more portions of information; and

generating a paper document wherein contents of the paper document comprise a visual representation of the information written by the user on the one or more sheets of paper and the visual representations generated for the one or more portions of information.

37. The computer program of claim 36 wherein the instructions for identifying the portion of information for each mark in the one or more marks comprise instructions for:

identifying an information capture device from the one or more information capture devices for the mark; and

selecting a portion of information captured by the information capture device identified for the mark.

38. The computer program of claim 37 wherein the instructions for identifying the information capture device for the mark comprise instructions for:

determining a type of information corresponding to the mark;

identifying a set of information capture devices from the one or more information capture devices that are configured to record the type of information corresponding to the mark; and

selecting an information capture device from the set of information capture devices for the mark.

39. The computer program of claim 36 wherein the instructions for generating the paper document comprise instructions for:

generating an electronic representation comprising a visual representation of the information written by the user and the visual representations generated for the one or more portions of information; and

printing the electronic representation on one or more paper pages to generate the paper document.

40. The computer program of claim 39 wherein the instructions for generating the electronic representation comprise instructions for:

determining a location in the visual representation of the information written by the user for placing each visual representation; and

generating the electronic representation such that each visual representation is placed in the visual representation of the information in the location determined for the visual representation.

41. The computer program of claim 36 wherein the instructions for generating the visual representation for each portion of information in the one or more portions of information comprise instructions for:

determining a type of information for each portion of information; and

generating the visual representation for a portion of information based upon the type of information determined for the portion of information.

42. The computer program of claim 36 wherein the one or more portions of information include a first portion comprising audio information and wherein the instructions for generating the visual representation comprise instructions for transcribing the audio information in the first portion.

43. The computer program of claim 36 wherein the one or more portions of information include a first portion comprising video information and wherein the instructions for generating the visual representation comprise instructions for selecting a keyframe from the video information of the first portion.

44. The computer program of claim 36 wherein the instructions for determining the one or more portions of information comprise instructions for:

identifying a signal that caused capture of information by a first information capture device from the one or more information capture devices; and

identifying information captured by the first information capture device in response to the signal.

45. The computer program of claim 44 wherein the contents of the paper document comprise a visual representation of the information captured by the first information capture device in response to the signal.

46. The computer program of claim 36 wherein the one or more portions of information includes a first portion and the instructions for determining the one or more portions of information comprise instructions for selecting the first portion when a threshold associated with a rule has been satisfied.