Searching documents for ranges of numeric values

• US 8,271,498 B2
• Filed: 08/12/2008
• Issued: 09/18/2012
• Est. Priority Date: 09/24/2004
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fontoura, Marcus Felipe, Lempel, Ronny, Qi, Runping, Zien, Jason Yeong
• Primary Examiner(s): LY, CHEYNE D
• Application Number:
  US12/190,495
• Publication Number:
  US 20080301130A1
• Time in Patent Office: 1,498 Days
• Field of Search: 707/715, 707/742
• US Class Current: 707/742
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

Provided are a method, system, and article of manufacture for searching documents for ranges of numeric values. Document identifiers for documents are accessed, wherein the documents include at least one value that is a member of a set of values. A number of posting lists are generated. Each posting list is associated with a range of consecutive values within the set of values and includes document identifiers for documents including at least one value within the range of consecutive values associated with the posting list, and wherein each document identifier is associated with one value in the set of values included in the document identified by the document identifier. The generated posting lists are stored, wherein the posting lists are used to process a query on a range of values within the set of values.

Claims

1. A computer implemented method, comprising:

accessing document identifiers for documents;

scanning the documents to determine values comprising searchable terms in the documents that are members of a set of values, wherein the set of values comprises at least one of an integer and a real number;

generating a number of posting lists, wherein each posting list is associated with a range of a number of consecutive values within the set of values, wherein the number of consecutive values is determined by dividing the number of values in the set of values by the number of positing lists, and wherein each posting list includes at least one of the document identifiers for at least one of the documents including at least one of the values within the range of consecutive values associated with the posting list; and

storing the generated posting lists, wherein the posting lists are used to process a query on a range of values within the set of values.

2. The method of claim 1, further comprising:

sorting the document identifiers and value pairs in each posting list by the document identifiers.

3. A computer implemented method, comprising:

accessing document identifiers for documents;

scanning the documents to determine values comprising searchable terms in the documents that are members of a set of values, wherein the set of values comprises at least one of an integer and a real number;

generating a number of posting lists associated with a first level, wherein each posting list is associated with a range of a number of consecutive values within the set of values, wherein the number of consecutive values is determined by dividing the number of values in the set of values by the number of positing lists, and wherein each posting list includes at least one of the document identifiers for at least one of the documents including at least one of the values within the range of consecutive values associated with the posting list; and

performing at least one iteration of generating posting lists for an additional level, wherein each posting list generated for the additional level is formed by merging at least two posting lists associated with a previous level, wherein each generated posting list at one additional level is associated with consecutive values in the set of values, wherein each document in the generated posting list at the additional level includes one value in the consecutive values associated with the posting list at the additional level, and wherein a new additional level and posting lists associated therewith are generated with each iteration.

4. The method of claim 3, wherein the number of posting lists generated for the first level is a function of a number of the values in the set of values and a maximum number of records to filter in posting lists at the first level.

5. The method of claim 3, wherein the function comprises dividing the number of values in the set of values by two times the maximum number of records to filter.

6. The method of claim 3, wherein each posting list generated for one additional level is formed by merging a fixed number of posting lists associated with the previous level.

7. The method of claim 6, wherein the fixed number of posting lists is a function of the number of posting lists generated for the first level and a number of levels for which posting lists are generated.

8. The method of claim 3, wherein the additional iterations are performed to generate posting lists for a number of levels.

9. The method of claim 3, further comprising:

filtering at least one of the determined posting lists at the first level in response to the determined posting lists at the first level including values outside of the query range of values to remove values not within the query range of values to form at least one filtered posting list only including values within the query range of values, wherein merging the determined posting lists comprises merging the at least one filtered posting list and determined posting lists that are not subject to filtering.

10. The method of claim 9, wherein merging the determined posting lists comprises:

accessing at least one of the document identifiers associated with a lowest value from the determined posting lists;

adding the accessed at least one of the document identifiers to the merged posting list; and

repeating the operations of accessing and adding the document identifiers for a next lowest value from the determined posting lists until all the document identifiers in the determined posting lists have been added to the merged posting list, wherein the document identifiers in the merged posting list are sorted according to the values associated with the document identifiers.

11. A system, comprising:

a processor; and

a computer readable medium including code executed by the processor to perform operations, the operations comprising;

accessing document identifiers for documents;

scanning the documents to determine values comprising searchable terms in the documents that are members of a set of values, wherein the set of values comprises at least one of an integer and a real number;

generating a number of posting lists, wherein each posting list is associated with a range of a number of consecutive values within the set of values, wherein the number of consecutive values is determined by dividing the number of values in the set of values by the number of positing lists, and wherein each posting list includes at least one of the document identifiers for at least one of the documents including at least one of the values within the range of consecutive values associated with the posting list; and

storing the generated posting lists, wherein the posting lists are used to process a query on a range of values within the set of values.

12. An article of manufacture comprising at least one of a hardware device implementing logic and a computer storage media having computer executable code to cause operations to be performed, the operations comprising:

accessing document identifiers for documents;

scanning the documents to determine values comprising searchable terms in the documents that are members of a set of values, wherein the set of values comprises at least one of an integer and a real number;

generating a number of posting lists, wherein each posting list is associated with a range of a number of consecutive values within the set of values, wherein the number of consecutive values is determined by dividing the number of values in the set of values by the number of positing lists, and wherein each posting list includes at least one of the document identifiers for at least one of the documents including at least one of the values within the range of consecutive values associated with the posting list; and

storing the generated posting lists, wherein the posting lists are used to process a query on a range of values within the set of values.

13. The article of manufacture of claim 12, wherein the operations further comprise:

sorting the document identifiers and value pairs in each posting list by the document identifiers.