Method and system for document fingerprint matching in a mixed media environment
First Claim
Patent Images
1. A method of image matching, comprising:
- receiving an image of at least part of a first media type;
generating a horizontal profile from the image, the horizontal profile identifying words in the image;
generating a plurality of bounding boxes, each bounding box surrounding a word in the horizontal profile;
horizontally classifying the plurality of bounding boxes in the image;
vertically classifying the plurality of bounding boxes in the image;
determining at least one spatial relationship between the plurality of bounding boxes by associating a first length of a first word with a second length of a second word in the horizontal profile and combining the horizontal and vertical classifications;
generating at least one horizontal grouping of bounding boxes and at least one vertical grouping of bounding boxes based on the spatial relationship;
generating a list of documents from a database of one or more documents, the list of documents including at least one common bounding box comprising an overlap of the at least one horizontal grouping of bounding boxes and the at least one vertical grouping of bounding boxes at a location in each document in the list;
determining a number of votes for each document in the list based on a number of common bounding boxes; and
identifying a matching document with a most number of votes from the list as a document containing the image.
1 Assignment
0 Petitions
Accused Products
Abstract
A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). In one particular embodiment, the MMR system provides for document fingerprint matching.
417 Citations
31 Claims
-
1. A method of image matching, comprising:
-
receiving an image of at least part of a first media type; generating a horizontal profile from the image, the horizontal profile identifying words in the image; generating a plurality of bounding boxes, each bounding box surrounding a word in the horizontal profile; horizontally classifying the plurality of bounding boxes in the image; vertically classifying the plurality of bounding boxes in the image; determining at least one spatial relationship between the plurality of bounding boxes by associating a first length of a first word with a second length of a second word in the horizontal profile and combining the horizontal and vertical classifications; generating at least one horizontal grouping of bounding boxes and at least one vertical grouping of bounding boxes based on the spatial relationship; generating a list of documents from a database of one or more documents, the list of documents including at least one common bounding box comprising an overlap of the at least one horizontal grouping of bounding boxes and the at least one vertical grouping of bounding boxes at a location in each document in the list; determining a number of votes for each document in the list based on a number of common bounding boxes; and identifying a matching document with a most number of votes from the list as a document containing the image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A system for image matching, comprising:
-
a database operable to store one or more documents; a processor; and a feature extraction module stored on a memory and executable by the processor, the feature extraction module operable to; receive an image of at least part of a first media type, generate a horizontal profile from the image, the horizontal profile identifying words in the image, generate a plurality of bounding boxes, each bounding box surrounding a word in the horizontal profile, horizontally classify the plurality of bounding boxes in the image, vertically classify the plurality of bounding boxes in the image, determine at least one spatial relationship between the plurality of bounding boxes by an association of a first length of a first word with a second length of a second word in the horizontal profile and a combination of the horizontal and vertical classifications, generate at least one horizontal grouping of bounding boxes and at least one vertical grouping of bounding boxes based on the spatial relationship, generate a list of documents from the database, the list of documents including at least one common bounding box comprising an overlap of the at least one horizontal grouping of bounding boxes and the at least one vertical grouping of bounding boxes at a location in each document in the list, determine a number of votes for each document in the list based on a number of common bounding boxes and identify a matching document with a most number of votes from the list as a document containing the image. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21)
-
-
22. A method of providing interaction between a first media type and a second media type, comprising:
-
receiving an image of at least part of the first media type; extracting a plurality of features from the image, including a plurality of words; generating a horizontal profile from the image, the horizontal profile including the words in the image; generating a plurality of bounding boxes, each bounding box surrounding a word in the horizontal profile; horizontally classifying the plurality of bounding boxes in the image; vertically classifying the plurality of bounding boxes in the image; determining at least one spatial relationship between the plurality of bounding boxes by associating a first length of a first word with a second length of a second word in the horizontal profile and combining the horizontal and vertical classifications; generating at least one horizontal grouping of bounding boxes and at least one vertical grouping of bounding boxes based on the spatial relationship; generating a list of documents from a database of one or more documents, the list of documents including at least one symbolic representation comprising an overlap of the at least one horizontal grouping of bounding boxes and the at least one vertical grouping of bounding boxes at a location in each document in the list; determining a number of votes for each document in the list based on a number of symbolic representations; identifying a matching document with a most number of votes as a document containing the image; and providing the document containing the image as the second media type. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30)
-
-
31. A computer program product comprising a computer readable non-transitory storage medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:
-
receive an image of at least part of a first media type; generate a horizontal profile from the image, the horizontal profile identifying words in the image; generate a plurality of bounding boxes, each bounding box surrounding a word in the horizontal profile; horizontally classify the plurality of bounding boxes in the image; vertically classify the plurality of bounding boxes in the image; determine at least one spatial relationship between the plurality of bounding boxes by an association of a first length of a first word with a second length of a second word in the horizontal profile and a combination the horizontal and vertical classifications; generate at least one horizontal grouping of bounding boxes and at least one vertical grouping of bounding boxes based on the spatial relationship; generate a list of documents from a database of one or more documents, the list of documents including at least one common bounding box comprising an overlap of the at least one horizontal grouping of bounding boxes and the at least one vertical grouping of bounding boxes at a location in each document in the list; determine a number of votes for each document in the list based on a number of common bounding boxes; and identify a matching document page with a most number of votes from the list as a document containing the image.
-
Specification