METHOD AND SYSTEM FOR SEARCHING NUMERICAL TERMS

• US 20100332502A1
• Filed: 06/30/2009
• Published: 12/30/2010
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cohen, Doron, Carmel, David, Sznajder, Benjamin
• Granted Patent: US 8,180,785 B2
• Time in Patent Office: Days
• US Class Current: 707/759
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/953 Querying, e.g. by the use of web search engines
• CPC Class Codes: G06F 40/205 Parsing

Abstract

A method and system for searching numerical terms are provided. The method includes processing a numerical term to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters identified as a number by a numeric parser, and indexing the numerical term in a search index by the stem for search retrieval. The method includes receiving a query and processing the query to divide it into terms, including a numerical term. The numerical terms of the query are also processed to a stem and a distance measurement and document results are retrieved for the stem from an index.

Claims

1. A method for searching numerical terms, comprising:

processing in a computer-based application a numerical term to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters identified as a number by a numeric parser;

indexing the numerical term in a search index by the stem for search retrieval;

wherein any of said steps are implemented in either of computer hardware or computer software and embodied in a computer-readable medium.

2. The method as claimed in claim 1, wherein processing a numerical term to a stem, includes interpreting number formats to approximate to the nearest stem.

3. The method as claimed in claim 1, wherein processing a numerical term to a stem includes a numerical stemming algorithm including a modulus set for the sensitivity of the stemming.

4. The method as claimed in claim 3, wherein a stem is a multiple of the modulus.

5. The method as claimed in claim 1, wherein a document is tokenized to determine textual terms and numerical terms in the document, and wherein textual terms are processed by a textual stemming algorithm and indexed by a stem for search retrieval.

6. The method as claimed in claim 1, wherein indexing of terms by a stem includes a reference to the document and offset in the document of a term.

7. The method as claimed in claim 1, wherein indexing of numerical terms includes the distance measurement to its stem.

8. The method as claimed in claim 1, including:

receiving a query and processing the query to divide it into terms, including numerical terms;

processing a numerical term of the query to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters identified as a number by a numeric parser; and

retrieving document results for the stem from an index.

9. The method as claimed in claim 8, wherein processing a numerical term includes expanding the term to more than one stem.

10. The method as claimed in claim 8, including scoring results by comparing the distance measurements of the query terms and the document results for the stem.

11. The method as claimed in claim 8, including scoring results by the similarity between two pieces of text including keywords and numbers, by expanding a cosine similarity measure by integrating distance measurement between numeric terms.

12. The method as claimed in claim 8, including providing an exact numeric search by filtering out terms with a different distance measurement from the query term.

13. The method as claimed in claim 8, including expanding a query term to cover a given range, and filtering out terms whose distance measurement exceeds the given range.

14. A computer program product for searching numerical terms, the computer program product comprising:

a computer readable medium;

computer program instructions operative to;

process a numerical term to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters identified as a number by a numeric parser;

index the numerical term in a search index by the stem for search retrieval;

wherein said program instructions are stored on said computer readable medium.

15. A method of providing a service to a customer over a network, the service comprising:

processing in a computer-based application a numerical term to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters identified as a number by a numeric parser;

indexing the numerical term in a search index by the stem for search retrieval;

wherein any of said steps are implemented in either of computer hardware or computer software and embodied in a computer-readable medium.

16. A system for searching numerical terms, comprising:

a processor;

a computer-based stemming module for processing a numerical term to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters including identified as a number by a numeric parser;

an indexer for indexing the numerical term in a search index by the stem for search retrieval;

wherein any of the stemming module and the indexer are implemented in either of computer hardware or computer software and embodied in a computer readable medium.

17. The system as claimed in claim 16, including a tokenizer for selecting numerical terms includes interpreting number formats.

18. The system as claimed in claim 16, including a user interface for setting a modulus for the sensitivity of the stemming.

19. The system as claimed in claim 16, wherein the tokenizer determines textual terms and numerical terms in a document, and wherein textual terms are processed by a textual stemming module.

20. The system as claimed in claim 16, including:

a query processor for processing a query to divide it into terms, including a numerical term and processing a numerical term of the query to a stem, the stem being in the form of a number, and a distance measurement of the numerical term to the stem, wherein a numerical term is a string of characters identified as a number by a numeric parser; and

a retriever for retrieving results for the stem from the index.

21. The system as claimed in claim 20, wherein including a query expander for processing a numerical term includes expanding the term to more than one stem.

22. The system as claimed in claim 20, including a scoring mechanism for scoring results by comparing the distance measurements of the query terms and the document results for the stem.

23. The system as claimed in claim 22, wherein the scoring mechanism includes scoring by the similarity between two pieces of text including keywords and numbers, by expanding a cosine similarity measure by integrating distance measurement between numeric terms

24. The system as claimed in claim 20, including a user input mechanism for requesting an exact numeric search, and an exact number filter for filtering out terms with a different distance measurement from the query term.

25. The system as claimed in claim 20, including a user input mechanism for requesting a number range search, and a number range filter for filtering out terms whose distance measurement exceeds the given range.