Data organization and access for mixed media document system
First Claim
1. A computer-implemented method for accessing information in a mixed media document system, the method comprising:
- receiving an image patch of a target document;
determining, with one or more processors, from the received image patch, a query that indicates a two-dimensional geometric relationship between a pair of document features in the target document, the two-dimensional geometric relationship including an indication that the pair of document features in the target document are a horizontally adjacent pair of document features or a vertically adjacent pair of document features;
comparing, with the one or more processors, the query to an index table of document features from mixed media documents to identify candidate regions in the mixed media documents that comprise the query, the index table comprising locations of the document features in the mixed media documents; and
responsive to comparing the query to the document features in the index table, identifying one or more of the mixed media documents comprising the identified candidate regions comprising the query by;
adding a weight to an array of an accumulator for each cell in a zone around each pair of document features based on an inverse document frequency associated with each pair of document features, the inverse document frequency being inversely proportional to a number of document pages that contain the image patch;
searching the array of the accumulator for a cell with a maximum value; and
in response to the maximum value exceeding a threshold, reporting coordinates of the cell as a location of the image patch.
1 Assignment
0 Petitions
Accused Products
Abstract
A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system includes a content-based retrieval database configured with an index table to represent two-dimensional geometric relationships between objects extracted from a printed document in a way that allows look-up using a text-based index. A ranked set of document, page and location hypotheses can be computed given data from the index table. The techniques effectively transform features detected in an image patch into textual terms (or other searchable features) that represent both the features themselves and the geometric relationship between them. A storage facility can be used to store additional characteristics about each document image patch.
-
Citations
18 Claims
-
1. A computer-implemented method for accessing information in a mixed media document system, the method comprising:
-
receiving an image patch of a target document; determining, with one or more processors, from the received image patch, a query that indicates a two-dimensional geometric relationship between a pair of document features in the target document, the two-dimensional geometric relationship including an indication that the pair of document features in the target document are a horizontally adjacent pair of document features or a vertically adjacent pair of document features; comparing, with the one or more processors, the query to an index table of document features from mixed media documents to identify candidate regions in the mixed media documents that comprise the query, the index table comprising locations of the document features in the mixed media documents; and responsive to comparing the query to the document features in the index table, identifying one or more of the mixed media documents comprising the identified candidate regions comprising the query by; adding a weight to an array of an accumulator for each cell in a zone around each pair of document features based on an inverse document frequency associated with each pair of document features, the inverse document frequency being inversely proportional to a number of document pages that contain the image patch; searching the array of the accumulator for a cell with a maximum value; and in response to the maximum value exceeding a threshold, reporting coordinates of the cell as a location of the image patch. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A non-transitory machine-readable medium encoded with instructions, that when executed by one or more processors, cause the one or more processors to carry out a process for accessing information in a mixed media document system, the process comprising:
-
receiving an image patch of a target document; determining, from the received image patch, a query that indicates a two-dimensional geometric relationship between document features in the target document, the two-dimensional geometric relationship including an indication that the document features in the target document are a horizontally adjacent pair of document features or a vertically adjacent pair of document features; comparing the query to an index table of the document features from mixed media documents to identify candidate regions in the mixed media documents that comprise the query, the index table comprising locations of the document features in the mixed media documents; and responsive to comparing the query to the document features in the index table, identifying one or more of the mixed media documents comprising the identified candidate regions comprising the query by; adding a weight to an array of an accumulator for each cell in a zone around each pair of document features based on an inverse document frequency associated with each pair of document features, the inverse document frequency being inversely proportional to a number of document pages that contain the image patch; searching the array of the accumulator for the cell with a maximum value; and in response to the maximum value exceeding a threshold, reporting coordinates of the cell as a location of the image patch. - View Dependent Claims (8, 9, 10, 11, 12)
-
-
13. A system comprising:
-
one or more processors; and a memory, the memory storing instructions, which when executed by the one or more processors, cause the one or more processors to; receive an image patch of a target document; determine, from the received image patch, a query that indicates a two-dimensional geometric relationship between document features in the target document, the two-dimensional geometric relationship including an indication that the document features in the target document are a horizontally adjacent pair of document features or a vertically adjacent pair of document features; compare the query to an index table of the document features from mixed media documents to identify candidate regions in the mixed media documents that comprise the query, the index table comprising locations of the document features in the mixed media documents; and responsive to comparing the query to the document features in the index table, identify one or more of the mixed media documents comprising the identified candidate regions comprising the query by; adding a weight to an array of an accumulator for each cell in a zone around each pair of document features based on an inverse document frequency associated with each pair of document features, the inverse document frequency being inversely proportional to a number of document pages that contain the image patch; searching the array of the accumulator for the cell with a maximum value; and in response to the maximum value exceeding a threshold, reporting coordinates of the cell as a location of the image patch. - View Dependent Claims (14, 15, 16, 17, 18)
-
Specification