SYSTEM AND ARTICLE OF MANUFACTURE FOR SEARCHING DOCUMENTS FOR RANGES OF NUMERIC VALUES

• US 20080294634A1
• Filed: 08/06/2008
• Published: 11/27/2008
• Est. Priority Date: 09/24/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fontoura, Marcus Felipe, Lempel, Ronny, Zien, Jason Yeong, Qi, Runping
• Granted Patent: US 8,346,759 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99937 Sorting

Abstract

Provided are a system and article of manufacture for searching documents for ranges of numeric values. Document identifiers for documents include at least one value that is a member of a set of values. A number of posting lists is generated, wherein each posting list is associated with a range of consecutive values within the set of values and includes document identifiers for documents including at least one value within the range of consecutive values associated with the posting list, and wherein each document identifier is associated with one value in the set of values included in the document identified by the document identifier. The generated posting lists are stored, wherein the posting lists are used to process a query on a range of values within the set of values. A query on a query range of values within the set of values is received and a determination is made of a minimum number of posting lists associated with consecutive values that together include the query range of values. The determined posting lists are merged to form a merged posting list including document identifiers of documents including values within the query range. The document identifiers in the merged posting list are returned.

Claims

1. A system in communication, comprising:

a processor;

a computer readable medium including code executed by the processor to perform operations, the operations comprising;

accessing document identifiers for documents, wherein the documents include at least one value that is a member of a set of values;

generating a number of posting lists, wherein each posting list is associated with a range of consecutive values within the set of values and includes document identifiers for documents including at least one value within the range of consecutive values associated with the posting list, and wherein each document identifier is associated with one value in the set of values included in the document identified by the document identifier;

storing the generated posting lists, wherein the posting lists are used to process a query on a range of values within the set of values;

receiving a query on a query range of values within the set of values;

determining a minimum number of posting lists associated with consecutive values that together include the query range of values;

merging the determined posting lists to form a merged posting list including document identifiers of documents including values within the query range; and

returning the document identifiers in the merged posting list.

2. The system of claim 1, wherein the operations further comprise:

sorting the document identifier and value pairs in each posting list by the document identifiers.

3. The system of claim 1, wherein the range associated with each posting list includes a number of values determined by dividing the number of values in the set of values by the number of posting lists.

4. The system of claim 1, wherein determining the minimum number of posting lists comprises determining a minimum number of posting lists including values outside of the query range of values that are filtered before merging the posting lists.

5. The system of claim 1, wherein the operations further comprise:

filtering at least one of the determined posting lists including values outside of the query range of values to remove values not within the query range of values to form at least one filtered posting list only including values within the query range of values, wherein merging the determined posting lists comprising merging the at least one filtered posting list and determined posting lists that are not subject to filtering.

6. The system of claim 5, wherein the merging operation comprises an OR merge operation.

16. The system of claim 1, wherein the set of values comprises at least one of an integer and a real number.

7. A system in communication, comprising:

a processor;

a computer readable medium including code executed by the processor to perform operations, the operations comprising;

accessing document identifiers for documents including at least one value that is a member of a set of values;

generating a number of posting lists associated with a first level, wherein each posting list is associated with a range of consecutive values within the set of values and includes document identifiers for documents including at least one value within the range of consecutive values associated with the posting list, and wherein each document identifier is associated with one value in the set of values included in the document identified by the document identifier;

performing at least one iteration of generating posting lists for an additional level, wherein each posting list generated for the additional level is formed by merging at least two posting lists associated with a previous level, wherein each generated posting list at one additional level is associated with consecutive values in the set of values, wherein each document in the generated posting list at the additional level includes one value in the consecutive values associated with the posting list at the additional level, and wherein a new additional level and posting lists associated therewith are generated with each iteration;

receiving a query on a query range of values within the set of values;

determining a minimum number of posting lists associated with one or more levels having consecutive values that include the query range of values;

merging the determined posting lists to form a merged posting list including document identifiers of documents including values within the query range; and

returning the document identifiers in the merged posting list.

8. The system of claim 7, wherein the number of posting lists generated for the first level is a function of a number of the values in the set of values and a maximum number of records to filter in posting lists at the first level.

9. The system of claim 8, wherein the function comprises dividing the number of values in the set of values by two times the maximum number of records to filter.

10. The system of claim 7, wherein each posting list generated for one additional level is formed by merging a fixed number of posting lists associated with the previous level.

11. The system of claim 10, wherein the fixed number of posting lists is a function of the number of posting lists generated for the first level and a number of levels for which posting lists are generated.

12. The system of claim 7, wherein the additional iterations are performed to generate posting lists for a number of levels.

13. The system of claim 7, wherein determining the minimum number of posting lists comprises determining a minimum number of posting lists including values outside of the query range of values that are filtered before merging the posting lists.

14. The system of claim 7, wherein the operations further comprise:

filtering at least one of the determined posting lists at the first level in response to the determined posting lists at the first level including values outside of the query range of values to remove values not within the query range of values to form at least one filtered posting list only including values within the query range of values, wherein merging the determined posting lists comprising merging the at least one filtered posting list and determined posting lists that are not subject to filtering.

15. The system of claim 14, wherein merging the determined posting lists comprises:

accessing documents identifiers associated with a lowest value from the determined posting lists;

adding the accessed document identifiers to the merged posting list; and

repeating the operations of accessing and adding the document identifiers for a next lowest value from the determined posting lists until all the document identifiers in the determined posting lists have been added to the merged posting list, wherein the document identifiers in the merged posting list are sorted according to the values associated with the document identifiers.

17. The system of claim 7, wherein the set of values comprises at least one of an integer and a real number.

18. The system of claim 7, wherein the number of posting lists to merge at level of posting lists is a function of a number of levels of posting lists and the number of posting lists for the first level.

19. An article of manufacture comprising at least one of a hardware device implementing logic and a computer storage media having computer executable code to cause operations to be performed, the operations comprising:

accessing document identifiers for documents, wherein the documents include at least one value that is a member of a set of values;

generating a number of posting lists, wherein each posting list is associated with a range of consecutive values within the set of values and includes document identifiers for documents including at least one value within the range of consecutive values associated with the posting list, and wherein each document identifier is associated with one value in the set of values included in the document identified by the document identifier;

storing the generated posting lists, wherein the posting lists are used to process a query on a range of values within the set of values;

receiving a query on a query range of values within the set of values;

determining a minimum number of posting lists associated with consecutive values that together include the query range of values;

merging the determined posting lists to form a merged posting list including document identifiers of documents including values within the query range; and

returning the document identifiers in the merged posting list.

20. The article of manufacture of claim 19, wherein the operations further comprise:

sorting the document identifier and value pairs in each posting list by the document identifiers.

21. The article of manufacture of claim 19, wherein the range associated with each posting list includes a number of values determined by dividing the number of values in the set of values by the number of posting lists.

22. The article of manufacture of claim 19, wherein determining the minimum number of posting lists comprises determining a minimum number of posting lists including values outside of the query range of values that are filtered before merging the posting lists.

23. The article of manufacture of claim 19, wherein the operations further comprise:

filtering at least one of the determined posting lists including values outside of the query range of values to remove values not within the query range of values to form at least one filtered posting list only including values within the query range of values, wherein merging the determined posting lists comprising merging the at least one filtered posting list and determined posting lists that are not subject to filtering.

24. The article of manufacture of claim 23, wherein the merging operation comprises an OR merge operation.

34. The article of manufacture of claim 19, wherein the set of values comprises at least one of an integer and a real number.

25. An article of manufacture comprising at least one of a hardware device implementing logic and a computer storage media having computer executable code to cause operations to be performed, the operations comprising:

accessing document identifiers for documents including at least one value that is a member of a set of values;

generating a number of posting lists associated with a first level, wherein each posting list is associated with a range of consecutive values within the set of values and includes document identifiers for documents including at least one value within the range of consecutive values associated with the posting list, and wherein each document identifier is associated with one value in the set of values included in the document identified by the document identifier;

performing at least one iteration of generating posting lists for an additional level, wherein each posting list generated for the additional level is formed by merging at least two posting lists associated with a previous level, wherein each generated posting list at one additional level is associated with consecutive values in the set of values, wherein each document in the generated posting list at the additional level includes one value in the consecutive values associated with the posting list at the additional level, and wherein a new additional level and posting lists associated therewith are generated with each iteration;

receiving a query on a query range of values within the set of values;

determining a minimum number of posting lists associated with one or more levels having consecutive values that include the query range of values;

merging the determined posting lists to form a merged posting list including document identifiers of documents including values within the query range; and

returning the document identifiers in the merged posting list.

26. The article of manufacture of claim 25, wherein the number of posting lists generated for the first level is a function of a number of the values in the set of values and a maximum number of records to filter in posting lists at the first level.

27. The article of manufacture of claim 26, wherein the function comprises dividing the number of values in the set of values by two times the maximum number of records to filter.

28. The article of manufacture of claim 25, wherein each posting list generated for one additional level is formed by merging a fixed number of posting lists associated with the previous level.

29. The article of manufacture of claim 28, wherein the fixed number of posting lists is a function of the number of posting lists generated for the first level and a number of levels for which posting lists are generated.

30. The article of manufacture of claim 25, wherein the additional iterations are performed to generate posting lists for a number of levels.

31. The article of manufacture of claim 25, wherein determining the minimum number of posting lists comprises determining a minimum number of posting lists including values outside of the query range of values that are filtered before merging the posting lists.

32. The article of manufacture of claim 25, wherein the operations further comprise:

filtering at least one of the determined posting lists at the first level in response to the determined posting lists at the first level including values outside of the query range of values to remove values not within the query range of values to form at least one filtered posting list only including values within the query range of values, wherein merging the determined posting lists comprising merging the at least one filtered posting list and determined posting lists that are not subject to filtering.

33. The article of manufacture of claim 32, wherein merging the determined posting lists comprises:

accessing documents identifiers associated with a lowest value from the determined posting lists;

adding the accessed document identifiers to the merged posting list; and

repeating the operations of accessing and adding the document identifiers for a next lowest value from the determined posting lists until all the document identifiers in the determined posting lists have been added to the merged posting list, wherein the document identifiers in the merged posting list are sorted according to the values associated with the document identifiers.

35. The article of manufacture of claim 25, wherein the set of values comprises at least one of an integer and a real number.

36. The article of manufacture of claim 25, wherein the number of posting lists to merge at level of posting lists is a function of a number of levels of posting lists and the number of posting lists for the first level.