System and Methods for Units-Based Numeric Information Retrieval

• US 20100332511A1
• Filed: 07/01/2009
• Published: 12/30/2010
• Est. Priority Date: 06/26/2009
• Status: Active Grant

• Current Assignee: Quantifind Incorporated
• Original Assignee: Entanglement Technologies, LLC
• Inventors: Stockton, John K., Tuchman, Ari K.
• Granted Patent: US 8,756,229 B2
• Time in Patent Office: Days
• US Class Current: 707/759
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/316 Indexing structures
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/3347 using vector based model
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06F 2216/11 Patent retrieval

Abstract

An information retrieval and analysis system for numeric data which provides high precision and recall for numeric search and uses a methodology for determining contextualization of the extracted data. The capabilities include extracting, parsing, and contextualizing numeric data including both a numeric value and an accompanying unit. This system facilitates the organization of largely unstructured numeric data into an inverted index and other database formats. An information retrieval system which enables the exploration and refinement of an extracted numeric data set defined by a search input that may be precise or initially vague. This system also facilitates analyzing and portraying numeric data graphically, creating knowledge by combining data from multiple sources, extracting correlations between seemingly disparate variables, and recognizing numeric data trends. This system uses local natural language processing, mathematical analysis, and expert-based scientific heuristics to score the numeric and contextual relevancy of the data to the query parameters.

Claims

1. An apparatus for information retrieval, comprising:

a computer system being programmed to receive a query from a remotely located user system via a network, the query specifying one or more numerical data constraints and contextual constraints;

said computer system including a memory storing a computer-searchable electronic index, the index comprising a plurality of entries, each of the index entries representing an item of numerical data extracted from at least one of a plurality of electronic source documents including one or more natural language documents;

said computer system being further programmed to determine the relevancy of the query to each of one or more of the index entries based at least partly on a comparison between the numerical data constraint and the index entry'"'"'s item of numerical data, and at least partly on a comparison between the contextual constraint and contextual information extracted from the corresponding source document from which the index entry'"'"'s item of numerical data was extracted.

2. The information retrieval apparatus of claim 1, wherein the item of numerical data comprises a unit of measure and a numerical value.

3. The information retrieval apparatus of claim 1, wherein the items of numerical data in the electronic source documents include product specifications for commercially available products.

4. The information retrieval apparatus of claim 3, wherein the computer system is further programmed to convert the extracted numerical data, from the original representation format used in the natural-language document from which said data was extracted, to a standard representation format for the index.

5. The information retrieval apparatus of claim 2, wherein at least one of the index entries comprises a compound unit of measure generated based on a correlation between two or more different units of measure associated with the items of numerical data.

6. The information retrieval apparatus of claim 1, wherein the index represents one or more of the items of numerical data in a plurality of levels of precision.

7. The information retrieval apparatus of claim 1, wherein one or more of the index entries is further associated with metadata stored in the index relating to an occurrence of said item of numerical data in the corresponding source document, including one or more of:

an original format of expression;

a number of significant numerical digits;

a frequency of occurrence in the source document;

contextual information extracted from the source document.

8. The information retrieval apparatus of claim 1, wherein the contextual information comprises information based on one or more keywords occurring in the corresponding source document, including position of said keywords within text of the source document.

9. The information retrieval apparatus of claim 8, wherein the contextual information comprises the keywords occurring in the corresponding source document and one or more additional terms automatically derived from said keywords.

10. The information retrieval apparatus of claim 1, wherein the extracted items of numerical data are error-checked by cross-checking an initially extracted item against one or more of (a) a distribution of similar extracted items and (b) one or more expert rules stored electronically in a knowledge library.

11. The information retrieval apparatus of claim 1, wherein the computer system is further programmed to export the electronic index into a reusable data structure in a standard format.

12. The information retrieval apparatus of claim 1, wherein the specified numerical data constraint comprises a unit of measure.

13. The information retrieval apparatus of claim 1, wherein the computer system is further programmed to convert, if necessary, the numerical data constraint specified in the query to a standard representation format of the index.

14. The information retrieval apparatus of claim 1, wherein the numerical constraints include one or more of:

a numeric value, a range of numeric values, and a specified functional relationship among a plurality of numerical constraint elements.

15. The information retrieval apparatus of claim 1, wherein the query specifies a plurality of numerical references of interest, each of the numerical references being defined in the query by a corresponding numerical constraint and contextual constraint.

16. The information retrieval apparatus of claim 1, wherein the contextual constraint in the query comprises one or more keywords.

17. The information retrieval apparatus of claim 1, wherein determining the relevancy includes computing a weighted sum of a plurality of scores.

18. The information retrieval apparatus of claim 2, wherein the computer system is further programmed to determine relevancy by computing whether the numerical data constraint specified by the query and the numerical value specified by the index entry match within a desired limit.

19. The information retrieval apparatus of claim 2, wherein the computer system is further programmed to determine relevancy by evaluating how well the numerical constraint specified by the query matches the numerical data specified in the index entry in light of a distribution of numerical values specified by a selected subset of the index entries.

20. The information retrieval apparatus of claim 1, wherein the comparison of the contextual constraints and the extracted contextual information takes into account one or more semantic synonyms.

21. The information retrieval apparatus of claim 1, wherein the comparison of the contextual constraints and the extracted contextual information takes into account one or more rules stored electronically in a knowledge library.

22. The information retrieval apparatus of claim 8, wherein the contextual constraint comprises one or more local keywords, and wherein the comparison of the contextual constraint with the extracted contextual information includes finding a match between the extracted contextual information and said local keywords, and takes into account a measure of proximity of said local key word match relative to the indexed item of numerical data within text of the source document.

23. The information retrieval apparatus of claim 22, wherein the measure of proximity reflects one or more of:

a number of words separating the represented item of numerical data from the local keyword match, and punctuation separating the represented item of numerical data from the local keyword match.

24. The information retrieval apparatus of claim 16, wherein the contextual constraint in the query comprises one or more global keywords, and the contextual information comprises one or more global themes of the source document.

25. The information retrieval apparatus of claim 24, wherein the global themes are extracted automatically by the computer system from the source document.

26. The information retrieval apparatus of claim 1, wherein the computer system programmed to determine relevancy is further programmed to perform Latent Semantic Analysis on at least some of the items of numerical data and the contextual information extracted from the source documents.

27. The information retrieval apparatus of claim 1, the computer system being further programmed to respond to the query by generating an interactive graph of the index entries determined to be most relevant to the query.

28. The information retrieval apparatus of claim 1 wherein the computer system comprises a plurality of computer servers located at one or more locations each geographically remote from the user system.

29. The information retrieval apparatus of claim 1 wherein the electronic source documents are accessible to the computer system via the network.

30. The information retrieval apparatus of claim 1 wherein the plurality of electronic source documents include a plurality of languages.

31. A computer memory apparatus storing a computer-searchable electronic index, the index comprising:

a plurality of searchable index entries, each of the entries representing an associated item of numerical data as a set of one or more tokens specifying a number and a unit of measure and being searchable thereby, the items of numerical data being extracted from a plurality of electronic source documents, each of the index entries being associated with an identifier for accessing the document from which the numerical data represented by said index entry was extracted;

information regarding the position of at least one of the extracted items of numerical data within text of the source document in which said item occurs; and

contextual information comprising one or more keywords occurring in the source documents, and further comprising information regarding the position of at least one of the keywords within text of the source document in which said keyword occurs.

32. A computer-automated method of information retrieval, comprising:

on a computing system, receiving a user-defined query specifying one or more numerical data constraints and contextual constraints, said query transmitted from a user system to the computing system via a network;

on said computing system, automatically determining the relevancy of the query to each of one or more entries in a computer-searchable electronic index, each of the index entries representing an item of numerical data extracted from at least one of a plurality of electronic source documents, including one or more natural language documents;

said determination of relevancy for each of the entries based at least partly on a comparison between the numerical data constraint and the entry'"'"'s item of numerical data, and at least partly on a comparison between the contextual constraint and contextual information extracted from the corresponding source document from which the entry'"'"'s item of numerical data was extracted;

on said computing system, generating a response to the query based on the relevancy determination;

transmitting the response from the computing system to the user system.

33. A computer-automated method of information retrieval, comprising:

on a user computing system, interactively specifying a query comprising one or more numerical data constraints and contextual constraints;

transmitting said query from the user system to a remote computing system via a network;

receiving a response to the query via the network from the remote computing system describing a plurality of responsive items of numerical data occurring in one or more of a plurality of electronic source documents, including one or more natural language documents, said responsive items determined by the remote system to be relevant to the query based at least partly on a comparison between the numerical data constraint and the items of numerical data, and at least partly on a comparison between the contextual constraint and contextual information extracted from the corresponding source document in which the item of numerical data occurred; and

displaying an interactive graph on the user system based on the response.

34. A computer apparatus for information retrieval, comprising:

a computer system being programmed to receive a user-defined query;

said computer system automatically determining the relevancy of the query to each of one or more entries in a computer-searchable electronic index, each of the index entries representing an item of data extracted from at least one of a plurality of electronic source documents;

said computer system being further programmed to generate a graph, based on the relevancy determination, comprising a separate data point for each of the index entries determined to be most relevant to the query, wherein a plurality of the most relevant index entries can be extracted from a single one of the source documents.

35. The information retrieval apparatus of claim 34, wherein said computer system is further programmed to provide an environment for interactively viewing the graph.

36. The information retrieval apparatus of claim 34, wherein one or more visual characteristics of each point on the graph are correlated with information descriptive of the corresponding index entry.

37. The information retrieval apparatus of claim 36, wherein the visual characteristics include one or more of color, size, shape, and transparency, and the descriptive information includes one or more of a relevancy score and source document for the corresponding index entry.

38. The information retrieval apparatus of claim 34, wherein the graph identifies one or more regions of the graph demarcated by one or more clustering algorithms.

39. The information retrieval apparatus of claim 38, wherein the regions of the graph are each labeled by one or more terms automatically extracted from the source documents for the relevant entries in each of the respective regions.

40. The information retrieval apparatus of claim 35, wherein the interactive environment is responsive to selection of a point on the graph by displaying content from the source document from which the item of numerical data corresponding to the selected point was extracted, at least some of said content located proximate to said item of numerical data within said source document.

41. The information retrieval apparatus of claim 35, said computer system is further programmed, in response to interactive selection of the graph data point corresponding to one of the index entries, to retrieve the source document from which the item of numerical data represented by said index entry was extracted.

42. The information retrieval apparatus of claim 35, wherein the interactive environment is operable to enable a user to provide feedback regarding the relevancy determination, and the method further includes modifying, in response to the feedback, one or more of:

the index, the query, a library of rules used in the determination of relevancy.

43. The information retrieval apparatus of claim 34, wherein one or more of the extracted items of data comprises a unit of measure and a numerical value.

44. A computer-automated method of information retrieval, comprising:

on a user computing system, interactively specifying a query comprising one or more numerical data constraints;

transmitting said query from the user system to a remote computing system via a network;

receiving a response to the query via the network from the remote computing system, said response defining a graph comprising a separate data point for each of a plurality of responsive items of numerical data occurring in one or more of a plurality of electronic source documents, said responsive items determined by the remote system to be relevant to the query and wherein a plurality of said relevant items can occur in a single one of the source documents;

interactively displaying the graph on the user system.