METHOD FOR USING DUAL INDICES TO SUPPORT QUERY EXPANSION, RELEVANCE/NON-RELEVANCE MODELS, BLIND/RELEVANCE FEEDBACK AND AN INTELLIGENT SEARCH INTERFACE

• US 20100205172A1
• Filed: 02/09/2009
• Published: 08/12/2010
• Est. Priority Date: 02/09/2009
• Status: Active Grant

• Current Assignee: Hong Kong Polytechnic University
• Original Assignee: Hong Kong Polytechnic University
• Inventors: LUK, Robert Wing Pong
• Granted Patent: US 8,620,900 B2
• Time in Patent Office: Days
• US Class Current: 707/715
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence

Abstract

A method for using dual indices to support query expansion, relevance/non-relevance models, blind/relevance feedback and an intelligent search interface, comprising using a computing device (89) to: access (101) an inverted index (103) to obtain an initial retrieval of results in response to a query, and to generate a rank list of the results, the results referring to information units (IUs) where the query occurs; and determine (105) a number of “N” IUs in the results that are assumed to be relevant by accessing a forward index (104); wherein the forward index (104) and inverted index (103) have pointers to locations in the IUs where terms of the query occur, and the forward index (104) retrieves a term frequency vector of the IU or a set of contexts of the IU.

Claims

1. A method for using dual indices to support query expansion, relevance/non-relevance models, blind/relevance feedback and an intelligent search interface, comprising using a computing device to:

access an inverted index to obtain an initial retrieval of results in response to a query, and to generate a rank list of the results, the results referring to information units (IUs) where terms of the query occur; and

determine a number of “

N”

IUs in the results that are assumed to be relevant by accessing a forward index;

wherein the forward index and inverted index have pointers to locations in the IUs where terms of the query occur more than once, and a forward index and inverted index pointer storage stores the locations in the IUs where the query term occurs only once in the IUs, and the forward index retrieves a term frequency vector of the IU or a set of contexts of the IU.

2. The method according to claim 1, wherein “

N”

is from 1 to 10000 or is query-dependent.

3. The method according to claim 2, wherein if “

N”

is query-dependent, the forward index is accessed one-by-one to determine the number “

N”

.

4. The method according to claim 1, wherein the forward index is searched by an information unit identifier (IID) of the IU, and is searched by terms of the query, and the frequencies of the query terms are collected.

5. The method according to claim 1, wherein the forward index is a set of frequency vectors for the IUs.

6. The method according to claim 1, wherein the inverted index and forward index are compressed.

7. The method according to claim 1, wherein the locations are cached in a cache that becomes a file.

8. The method according to claim 7, wherein the cache, inverted index and forward index are stored on solid state disk drives.

9. The method according to claim 1, wherein the forward index is any one from the group consisting of:

variable bit-block compression signature, superimposed coding signature, vocabulary index, trie, B-tree, heap B-tree, red-black tree, suffix arrays, suffix tree, PATRICIA trie, string B-tree, and DAWGs.

10. The method according to claim 4, further comprising the initial steps of:

storing the frequencies of the query terms in the forward index; and

storing the pointers to word locations in the forward index.

11. A system for supporting query expansion, relevance/non-relevance models, blind/relevance feedback and an intelligent search interface using dual indices, the system comprising:

a retrieval module to access an inverted index to obtain an initial retrieval of results in response to a query, and to generate a rank list of the results, the results referring to information units (IUs) where the query occurs; and

a post-processing module to determine a number of “

N”

IUs in the results that are assumed to be relevant by accessing a forward index;

wherein the forward index and inverted index have pointers to locations in the IUs where terms of the query occur more than once, and a forward index and inverted index pointer storage stores the locations in the IUs where the query term occurs only once in the IUs, and the forward index retrieves a term frequency vector of the IU or a set of contexts of the IU.

12. The system according to claim 11, further comprising solid state disk drives to store the inverted index and forward index.

13. A search engine providing support for query expansion, relevance/non-relevance models, blind/relevance feedback and an intelligent search interface, the search engine comprising:

dual indices consisting of an inverted index and a forward index;

a retrieval module to access an inverted index to obtain an initial retrieval of results in response to a query, and to generate a rank list of the results, the results referring to information units (IUs) where the query occurs; and

a post-processing module to determine a number of “

N”

IUs in the results that are assumed to be relevant by accessing a forward index;

wherein the forward index and inverted index have pointers to locations in the IUs where terms of the query occur more than once, and a forward index and inverted index pointer storage stores the locations in the IUs where the query term occurs only once in the IUs, and the forward index retrieves a term frequency vector of the IU or a set of contexts of the IU.