Method and System for Multi-Tier Image Matching in a Mixed Media Environment

• US 20060262976A1
• Filed: 07/31/2006
• Published: 11/23/2006
• Est. Priority Date: 10/01/2004
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Piersol, Kurt, Hart, Peter E., Hull, Jonathan J., Erol, Berna
• Granted Patent: US 8,521,737 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 21/78 to assure secure storage of data address-based protection against unauthorised use of memory G06F12/14; record carriers for use with machines and with at least a part designed to carry digital markings G06K19/00
• CPC Class Codes: G06F 21/80 in storage media based on magnetic or optical technology, e.g. disks with sectors preventing unauthorised reproduction or copying of disc-type recordable media G11B20/00
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images

Abstract

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system provides for multi-tier image matching.

Claims

1. A method of image matching, comprising:

capturing an image of at least part of a first media type;

determining whether the image contains a culling feature;

selecting a set of pages from a collection of documents in response to the determination;

capturing another image of at least part of the media type; and

matching the another image against the set of pages.

2. The method of claim 1, wherein the first media type is a paper document.

3. The method of claim 1, wherein the culling feature comprises at least one selected from the group consisting of a numeric-only bar code, an alphanumeric bar code, a 2-dimensional bar code, a graphic, an image, a trademark, a color, a combination of colors, a keyword, a logo, and a phrase.

4. The method of claim 1, wherein the set of pages is associated with the culling feature.

5. The method of claim 1, wherein the matching comprises:

extracting a feature from the captured another image; and

comparing the extracted feature with features contained in the set of pages.

6. The method of claim 1, further comprising:

providing a second media type in response to matching the another image.

7. The method of claim 6, wherein the second media type comprises at least one selected from the group consisting of a data structure, a command, text, audio, video, an image, a digital photograph, web link text, an application file, updated information, and services.

8. The method of claim 1, further comprising:

performing an action in response to matching the another image, wherein the action comprises at least one selected from the group consisting of retrieving information, placing an order, retrieving a video, retrieving a sound, storing information, creating a new document, printing a document or image, displaying a document or image, searching information, and presenting information.

9. A system for image matching, comprising:

a capture device operable to capture a first image of at least part of a first media type, the first image containing a culling feature; and

a database operable to select a set of pages within the database based on the culling feature,

wherein the capture device is further operable to capture a second image of at least part of the first media type, and

wherein the database is further operable to match the second image against the set of pages.

10. The system of claim 9, wherein the capture device comprises one selected from the group consisting of a cellular camera phone, a Personal Digital Assistant (PDA) device, a digital camera, a barcode reader, a radio frequency identification (RFID) reader, a computer peripheral, a web camera, and a video card.

11. The system of claim 9, wherein the first media type is a paper document.

12. The system of claim 9, wherein the culling feature comprises at least one selected from the group consisting of a numeric-only bar code, an alphanumeric bar code, a 2-dimensional bar code, a graphic, an image, a trademark, a color, a combination of colors, a keyword, a logo, and a phrase.

13. The system of claim 9, further comprising:

a processor operable to cause an action in response to matching the second image against the set of pages.

14. The system of claim 13, wherein the action comprises at least one selected from the group consisting of retrieving information, placing an order, retrieving a video, retrieving a sound, storing information, creating a new document, printing a document or image, displaying a document or image, searching information, and presenting information.

15. The system of claim 9, wherein the capture device is further operable to at least one of receive and output a second media type in response to matching the second image against the set of pages.

16. The system of claim 15, wherein the second media type comprises at least one selected from the group consisting of a data structure, a command, text, audio, video, an image, a digital photograph, web link text, an application file, updated information, and services.

17. The system of claim 15, wherein the database comprises:

a base media storage operable to store a collection of documents; and

a mixed media storage operable to store the second media type.

18. The system of claim 9, wherein the capture device is further operable to detect the culling feature in the captured first image.

19. The system of claim 9, wherein the database is further operable to detect the culling feature in the captured first image.

20. A method of interacting a first media type with a second media type, comprising:

capturing with a capture device a first image of at least part of the first media type;

detecting a culling feature contained in the captured first image;

selecting a set of pages within a document database in response to detecting the culling feature, the set of pages being associated with the culling feature;

capturing with the capture device a second image of at least part of the first media type;

matching the second image against the set of pages; and

identifying a second media type in response to positively matching the second image.

21. The method of claim 20, wherein the first media type is a paper document.

22. The method of claim 20, wherein the culling feature comprises at least one selected from the group consisting of a numeric-only bar code, an alphanumeric bar code, a 2-dimensional bar code, a graphic, an image, a trademark, a color, a combination of colors, a keyword, a logo, and a phrase.

23. The method of claim 20, wherein the second media type comprises at least one selected from the group consisting of a data structure, a command, text, audio, video, an image, a digital photograph, web link text, an application file, updated information, and services.

24. The method of claim 20, further comprising:

performing an action in response to matching the second image, wherein the action comprises at least one selected from the group consisting of retrieving information, placing an order, retrieving a video, retrieving a sound, storing information, creating a new document, printing a document or image, displaying a document or image, searching information, presenting information, and providing the second media type to the capture device.