Scalable lookup-driven entity extraction from indexed document collections
First Claim
1. A method for filtering a set of documents, comprising:
- receiving a list of entity strings;
determining a set of token sets that covers the entity strings in the list;
querying an inverted index generated on a first set of documents using the set of token sets to determine a set of document identifiers for a subset of the documents in the first set;
retrieving from the first set of documents a second set of documents identified by the set of document identifiers; and
filtering the second set of documents to include one or more documents of the second set that each include a match with at least one entity string of the list of entity strings.
2 Assignments
0 Petitions
Accused Products
Abstract
A set of documents is filtered for entity extraction. A list of entity strings is received. A set of token sets that covers the entity strings in the list is determined. An inverted index generated on a first set of documents is queried using the set of token sets to determine a set of document identifiers for a subset of the documents in the first set. A second set of documents identified by the set of document identifiers is retrieved from the first set of documents. The second set of documents is filtered to include one or more documents of the second set that each includes a match with at least one entity string of the list of entity strings. Entity recognition may be performed on the filtered second set of documents.
-
Citations
21 Claims
-
1. A method for filtering a set of documents, comprising:
-
receiving a list of entity strings; determining a set of token sets that covers the entity strings in the list; querying an inverted index generated on a first set of documents using the set of token sets to determine a set of document identifiers for a subset of the documents in the first set; retrieving from the first set of documents a second set of documents identified by the set of document identifiers; and filtering the second set of documents to include one or more documents of the second set that each include a match with at least one entity string of the list of entity strings. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A system for filtering a set of documents, comprising:
-
a document identifier filter that includes a covering token set determiner and an inverted index querier, wherein the covering token set determiner is configured to receive a list of entity strings and to determine a set of token sets that covers the entity strings in the list, and the inverted index querier is configured to query an inverted index generated on a first set of documents using the set of token sets to determine a set of document identifiers for a subset of the documents in the first set; a document retriever configured to retrieve from the first set of documents a second set of documents identified by the set of document identifiers; and an entity string matcher configured to filter the second set of documents to include one or more documents of the second set that each include a match with at least one entity string of the list of entity strings. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
-
17. A method for ad-hoc entity extraction, comprising:
-
filtering a first set of documents to generate a second set of documents that includes documents of the first set having a match with at least one entity string in a list of entity strings; and performing entity recognition on the second set of documents. - View Dependent Claims (18, 19, 20, 21)
-
Specification