User Interface for Mixed Media Reality

• US 20070047816A1
• Filed: 07/31/2006
• Published: 03/01/2007
• Est. Priority Date: 08/23/2005
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey, Erol, Berna, Hart, Peter, Hull, Jonathan
• Granted Patent: US 8,156,427 B2
• Time in Patent Office: Days
• US Class Current: 382/181
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one embodiment, an MMR document is retrieved based on recognition of a paper document. Responsive to a comparison of the paper document and a virtual multimedia document, a set of actions is displayed to a user. Responsive to a user selection, the user-selected action is performed. In another embodiment, a captured list of names is used to assist a user with labeling a media file.

Claims

1. A system for accessing mixed media documents, the system comprising:

a capture device for acquiring a first document; and

a mixed media processor for retrieving a second document and comparing the first document and the second document; and

a display device for displaying actions based on the comparison.

2. The system of claim 1, further comprising a communication mechanism for coupling the capture device to the mixed media processor.

3. The system of claim 2, where in the communication mechanism is one from the group of a network, a wired coupling, a infrared link, a Bluetooth link, a wireless communication link, a cellular communication link, a data communication link, a mobile data service, a Global System for Mobile Communications, and a public switched telephone network.

4. The system of claim 1, wherein the first document is a paper document.

5. The system of claim 1, wherein the capture device includes a portable input device capable of capturing digital data or an image representing a portion of the first document.

6. The system of claim 1, wherein the capture device is one from the group of a cellular camera phone;

a Personal Digital Assistant device;

a digital camera;

a barcode reader;

a radio frequency identification (RFID) reader;

a computer peripheral;

a web camera;

or a video card.

7. The system of claim 1, wherein the second document comprises a mixed media document.

8. The system of claim 1, wherein the display device is configured to display document information based on the comparison.

9. The system of claim 1, further comprising:

an action processor for, responsive to a user selection, performing the user-selected action.

10. The system of claim 9, wherein the user-selected action comprises a grab action.

11. The system of claim 10, wherein, responsive to the user selection, the action processor is configured to store an identifier of the second document.

12. The system of claim 10, wherein, responsive to the user selection, the action processor is configured to store an electronic representation of the first document.

13. The system of claim 9, wherein the user-selected action comprises a schedule action.

14. The system of claim 13, wherein, responsive to the user selection, the action processor is configured to store appointment data.

15. The system of claim 13, wherein, responsive to the user selection, the action processor is configured to create an appointment on an electronic calendar.

16. The system of claim 13, wherein, responsive to the user selection, the action processor is configured to schedule a recording on a digital video recorder.

17. The system of claim 9, wherein the user-selected action comprises a purchase action.

18. The system of claim 17, wherein, responsive to the user selection, the action processor is configured to place an order for a product.

19. The system of claim 17, wherein, responsive to the user selection, the action processor is configured to debit a user account.

20. The system of claim 17, wherein responsive to the user selection, the action processor is configured to schedule a transfer of a media file to a portable device.

21. A system for labeling a media file, the system comprising:

a document parser for capturing a list of names;

a display device for displaying the list of names to a user; and

a media file storage system for, responsive to a user selection, associating the media file with the user-selected name.

22. The system of claim 21, further comprising:

a print driver capture module for capturing a document, wherein the document parser is configured to parse the captured document.

23. The system of claim 21, further comprising:

an event capture module for capturing a document, wherein the document parser is configured to parse the captured document.

24. The system of claim 21, further comprising:

a scanner for capturing a document, wherein the document parser is configured to parse the captured document.

25. The system of claim 21, further comprising:

a name prioritizer for ordering the list of names.

26. An electronically-implemented method for accessing mixed media documents, the method comprising:

acquiring a first document;

retrieving a second document;

comparing the first document and the second document; and

displaying actions based on the comparison.

27. The method of claim 26, wherein the second document comprises a mixed media document.

28. The method of claim 26, wherein the first document is one from the group of a representation of the entire document, a representation of patch of text, a representation of single word, a representation of a section of an image, and any other representation of a matchable portion of the first document.

29. The method of claim 26, wherein acquiring a first document comprises one from the group of scanning a patch of text, scanning a patch of text and preprocessing to determine the likelihood that an intended feature description is extracted correctly, scanning a machine-readable identifier, inputting data that identifies the document, receiving email with the document attached, receiving the document by file transfer, scanning a portion of an image, and inputting the document.

30. The method of claim 26, wherein comparing the first document and the second document further comprises performing pattern matching between the first document and the second document.

31. The method of claim 26, further comprising:

displaying document information related to the second document based on the comparison.

32. The method of claim 26, further comprising:

responsive to a user selection, performing the user-selected action.

33. The method of claim 32, wherein the user-selected action comprises a grab action.

34. The method of claim 33, wherein performing the user-selected action comprises storing an identifier of the second document.

35. The method of claim 33, wherein performing the user-selected action comprises storing an electronic representation of the first document.

36. The method of claim 32, wherein the user-selected action comprises a schedule action.

37. The method of claim 36, wherein performing the user-selected action comprises storing appointment data.

38. The method of claim 36, wherein performing the user-selected action comprises creating an appointment on an electronic calendar.

39. The method of claim 36, wherein performing the user-selected action comprises scheduling a recording on a digital video recorder.

40. The method of claim 32, wherein the user-selected action comprises a purchase action.

41. The method of claim 40, wherein performing the user-selected action comprises placing an order for a product.

42. The method of claim 40, wherein performing the user-selected action comprises debiting a user account.

43. The method of claim 40, wherein performing the user-selected action comprises scheduling a transfer of a media file to a portable device.

44. A method for labeling a media file, the method comprising:

capturing a list of names;

displaying the list of names to a user; and

responsive to a user selection, associating the media file with the user-selected name.

45. The method of claim 44, wherein the list of names is captured from a source file.

46. The method of claim 45, wherein the list of names is captured in response to printing the source file.

47. The method of claim 45, wherein the list of names is captured in response to viewing the source file.

48. The method of claim 45, wherein the source file is one from the group of a text document, a web page, a presentation, and a spreadsheet.

49. The method of claim 44, wherein capturing a list of names comprises scanning a paper document.

50. The method of claim 44, wherein the media file is one from the group of an audio file, a video file, an image file, a document, a web page, a presentation, and a spreadsheet.

51. The method of claim 44, further comprising displaying a description of the media file.

52. The method of claim 44, wherein displaying the list of names further comprises ordering the list of names.

53. The method of claim 44, wherein the media file includes a metadata field, and wherein associating the media file with the user-selected name comprises storing the user-selected name in the metadata field.

54. The method of claim 44, wherein associating the media file with the user-selected name comprises storing the user-selected name as a file name of the media file.

55. The method of claim 44, wherein associating the media file with the user-selected name comprises the storing media file in a folder, wherein the folder is associated with a folder name including the user-selected name.