Authoring tools using a mixed media environment

• US 7,639,387 B2
• Filed: 07/31/2006
• Issued: 12/29/2009
• Est. Priority Date: 08/23/2005
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Graham, Jamey, Piersol, Kurt, Hart, Peter E., Hull, Jonathan J., Erol, Berna
• Primary Examiner(s): Zimmerman; Mark K
• Assistant Examiner(s): Chen; Huo Long
• Application Number:
  US11/461,268
• Publication Number:
  US 20070047781A1
• Time in Patent Office: 1,247 Days
• Field of Search: 358/1.18, 715/202, 715/201, 715/209
• US Class Current: 358/1.18
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition

Abstract

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system provides a method to associate related documents. The method acquires two associated documents and associates the related portions together. When the method acquires a portion of one of the document, it identifies the portion acquired, and provides the related portion of the other document.

Claims

1. A method for providing a mixed media environment for associating actions with acquired images, the method comprising:

using a computer system to perform the steps of;

acquiring an image of a machine-readable trademark or logotype associated with a company;

acquiring an action to be associated with the machine-readable trademark or logotype associated with the company;

indexing the acquired image to associate the machine-readable trademark or logotype with the action to create a mixed media reality document, the mixed media reality document comprising the acquired image of the machine-readable trademark or logotype, the acquired action, and the index associating the action with the machine-readable trademark or logotype; and

storing the mixed media reality document for comparison to input images, wherein an input image comprising content matching the machine-readable trademark or logotype triggers the action.

2. The method of claim 1, wherein the acquiring the image comprises automatically capturing an image layout of a printed document upon printing.

3. The method of claim 2, wherein the automatically capturing is performed by one from the group consisting of a printed document capture module, an operating system, a print driver, a scanner, and a file transfer mechanism.

4. The method of claim 1, wherein acquiring the image comprises one from the group consisting of scanning a patch of text, scanning a patch of text and preprocessing to determine the likelihood that an intended feature description is extracted correctly, scanning a machine-readable identifier, inputting data that identifies a document, receiving email with a document attached, receiving a document by file transfer, scanning a portion of an image, and inputting a document.

5. The method of claim 1, wherein the action is one from the group consisting of retrieving information, writing information to a location, performing a search for information, displaying information on a client device, conducting an interactive dialog, deferring the action for later execution, modifying a document, purchase of a product, and input of a command to another system.

6. The method of claim 1, further comprising:

determining that the image is unique.

11. The method of claim 1, wherein the trademark or logotype is discernable by a camera but not by a human eye.

7. A method for providing a mixed media environment for annotating documents, the method comprising:

using a computer system to perform the steps of;

acquiring an image of a document of a first media type;

selecting a portion of the image of the document as a hotspot for adding an annotation associated with the portion;

acquiring data of a second media type as the annotation;

indexing the hotspot portion of the image of the document and the data to create a mixed media reality document, the mixed media reality document comprising the acquired image of the document, the acquired data, and the index associating the data with the hotspot portion of the image of the document; and

storing in a database the mixed media reality document for comparison to input images, wherein an input image comprising content matching the hotspot portion of the image triggers display of the associated data as the annotation.

8. The method of claim 7, wherein the acquiring the data of the second media type comprises retrieving contents from the Internet.

12. The method of claim 7, further comprising:

determining whether a different version of the document is stored in the database;

in response to a determination that the different version of the document is stored in the database, creating an intermediate linking layer linking locations in a version associated with the acquired representation with corresponding locations in the different version.

13. The method of claim 7, wherein display of the associated data as the annotation comprises launching a web browser directed to a webpage including the annotation.

14. The method of claim 7, wherein an input image matching the acquired representation of the document also triggers opening an application enabling addition of user annotations for association with the hotspot portion of the representation.

9. A method for providing a mixed media environment for associating documents, the method comprising:

using a computer system to perform the steps of;

acquiring a first representation of a first document;

selecting a first portion of the first representation of the first document as a first hotspot for associating with a second representation of a second document;

acquiring the second representation of the second document;

indexing the first hotspot portion of the first representation and the second representation to create a first mixed media reality document comprising the first representation, the second representation, and the index associating the second representation with the first hotspot portion of the first representation; and

storing in a database the first mixed media reality document for comparison to input images, wherein a first input image matching the first hotspot portion of the first representation triggers display of the second representation.

10. The method of claim 9, further comprising:

selecting a second portion of the second representation of the second document as a second hotspot for associating with the first representation of the first document;

indexing the second hotspot portion of the second and the first representation to create a second mixed media reality document comprising the second representation, the first representation, and the indexing associating the first representation with the second hotspot portion of the second representation; and

storing in the database the second mixed media reality document for comparison to the input images, wherein a second input image matching the second hotspot portion of the second representation triggers display of the first representation.

15. The method of claim 9, wherein the first document and the second document are pages within a larger document.

16. The method of claim 9, wherein the second document comprises subject matter related to subject matter associated with the first document.