System And Methods For Creation And Use Of A Mixed Media Environment With Geographic Location Information

• US 20070047782A1
• Filed: 07/31/2006
• Published: 03/01/2007
• Est. Priority Date: 08/23/2005
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey, Piersol, Kurt, Hull, Jonathan, Van Olst, Daniel
• Granted Patent: US 8,005,831 B2
• Time in Patent Office: Days
• US Class Current: 382/124
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system provides a user with information related to the location of the user. The system acquires a document, extracts location related information from the document, identifies the location of the users, and provides information to the user based on the identified location and the extracted information.

Claims

1. A method for providing a mixed media environment for interacting with documents of a first media type and a second media type, the method comprising:

generating a representation of a document of a first media type;

indexing the generated representation of the document to a mixed media document;

identifying a first geographic location based on the document of the first media type;

identifying a second geographic location;

performing an action specified by the mixed media document based on the first and second geographic locations.

2. The method of claim 1, wherein the generating the representation of the document of the first media type comprises automatically capturing a text layout of a printed document.

3. The method of claim 2, wherein the automatically capturing is done by one from a group of a printed document capture module, an operating system, a print driver, a scanner, a file transfer mechanism.

4. The method of claim 1, wherein the generated representation is one from the group of a representation of the entire document, a representation of patch of text, a representation of single word, a representation of a section of an image, and any other representation of a matchable portion of the document.

5. The method of claim 1, wherein indexing the generated representation includes association of one from the group of an index, an electronic representation of the document, a second media type, and an action, with the generated representation.

6. The method of claim 1, wherein indexing the generated representation includes one from the group of specifying x-y locations of characters of a printed document, specifying x-y locations of words of the printed document, specifying x-y locations of an image or a portion of an image in a printed document, performing an OCR operation and specifying x-y locations of characters and/or words, performing feature extraction and specifying x-y locations of characters and/or words.

7. The method of claim 1, wherein identifying the first geographic location comprises parsing the document for addresses and entity names representing locations.

8. The method of claim 1, wherein identifying the first geographic location comprises parsing HTML tags for the first geographic location.

9. The method of claim 1, wherein identifying the second geographic location comprises determining the geographic location using a GPS device.

10. The method of claim 1, wherein performing an action specified by the mixed media document comprises providing directional information from the second geographic location to the first geographic location.

11. The method of claim 1, wherein the action is one from the group of retrieving information, writing information to a location, performing a search for information, displaying information on a client device, conducting an interactive dialog, deferring the action for later execution, modifying a document, purchase of a product, and input of a command to another system.

12. The method of claim 1, wherein performing an action specified by the mixed media document comprises providing advertisements related to the first and second geographic location.

13. The method of claim 12, wherein the advertisements are selected based on the content of the document.

14. The method of claim 1, wherein performing an action specified by the mixed media document comprises annotating a data structure based on the first geographic location.

15. A method for providing a mixed media environment for interacting with documents of a first media type and a second media type, the method comprising:

generating a representation of a document of a first media type;

parsing the document for an action of a second media type;

indexing the generated representation of the document to a mixed media document, the generated representation being associated with the action; and

performing the action.

16. The method of claim 15, wherein parsing the document further comprises:

parsing the document for an action of a second media type and an associated condition, and

wherein performing the action further comprises;

responsive to the associate condition being satisfied, performing the action.

17. The method of claim 15, wherein parsing the document further comprises parsing HTML tags for the action of the second media type.