Method And System For Position-Based Image Matching In A Mixed Media Environment

• US 20060262962A1
• Filed: 07/31/2006
• Published: 11/23/2006
• Est. Priority Date: 10/01/2004
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Piersol, Kurt, Hart, Peter E., Hull, Jonathan J., Erol, Berna, Lee, Dar-Shyang
• Granted Patent: US 8,332,401 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 21/80 in storage media based on magnetic or optical technology, e.g. disks with sectors preventing unauthorised reproduction or copying of disc-type recordable media G11B20/00
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images

Abstract

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system provides for position-based image matching.

Claims

1. A method of image matching, comprising:

capturing with a capture device a first image of at least part of a first media type;

capturing with the capture device a second image of at least part of the first media type;

tracking a position of the capture device over the first media type based on the first image and the second image; and

identifying a document page containing the first image and the second image dependent on the tracked position.

2. The method of claim 1, wherein the first media type is a paper document.

3. The method of claim 1, further comprising:

associating a second media type in response to the identification.

4. The method of claim 1, wherein the second media type comprises at least one selected from the group consisting of a data structure, a command, text, audio, video, an image, a digital photograph, web link text, an application file, updated information, and services.

5. The method of claim 1, further comprising:

maintaining a history of tracking the position of the capture device over the first media type.

6. The method of claim 1, further comprising:

predicting the position of the capture device over the first media type.

7. The method of claim 1, further comprising:

performing an action based on the identification, wherein the action comprises at least one selected from the group consisting of

retrieving information, placing an order, retrieving a video,

retrieving a sound, storing information, creating a new document,

printing a document or image, displaying a document or image,

searching information, and presenting information.

8. A system for image matching, comprising:

a capture device operable to capture a sequence of images of at least parts of a first media type;

a position tracking module operable to track a position of the capture device over the first media type based on the captured sequence of images; and

a database operable to store a collection of document pages against which the captured sequence of images is matchable dependent on the tracked position to identify at least one document page and location therein containing the sequence of images.

9. The system of claim 8, wherein the first media type is a paper document.

10. The system of claim 8, wherein a second media type is associated with the identification.

11. The system of claim 10, wherein the second media type comprises at least one selected from the group consisting of a data structure, a command, text, audio, video, an image, a digital photograph, web link text, an application file, updated information, and services.

12. The system of claim 8, wherein the capture device comprises one selected from the group consisting of a cellular camera phone, a Personal Digital Assistant (PDA) device, a digital camera, a barcode reader, a radio frequency identification (RFID) reader, a computer peripheral, a web camera, and a video card.

13. The system of claim 8, wherein the position tracking module is further operable to:

maintain a history of tracking the position of the capture device over the first media type.

14. The system of claim 8, wherein the position tracking module is further operable to:

predict the position of the capture device over the first media type.

15. A method of interacting a first media type with a second media type, comprising:

capturing with a capture device a sequence of images of at least parts of a first media type;

matching the captured sequence of images against a collection of document pages; and

tracking a position of the capture device over the first media type, wherein the tracking is constrained by the collection of document pages;

identifying at least one document page containing the captured sequence of images in response to the tracking and matching; and

providing a second media type to the capture device in response to the identification.

16. The method of claim 15, wherein the first media type is a paper document.

17. The method of claim 15, wherein the second media type comprises at least one selected from the group consisting of a data structure, a command, text, audio, video, an image, a digital photograph, web link text, an application file, updated information, and services.

18. The method of claim 15, further comprising:

performing an action based on the identification, wherein the action comprises at least one selected from the group consisting of

retrieving information, placing an order, retrieving a video,

retrieving a sound, storing information, creating a new document,

printing a document or image, displaying a document or image,

searching information, and presenting information.

19. The method of claim 15, further comprising:

maintaining a history of tracking the position of the capture device over the first media type.

20. The method of claim 15, further comprising:

predicting the position of the capture device over the first media type based on the captured sequence of images.