Method for organizing records of database search activity by topical relevance

• US 20030014399A1
• Filed: 03/12/2002
• Published: 01/16/2003
• Est. Priority Date: 03/12/2001
• Status: Active Grant

• Current Assignee: Nokia of America Corporation (Nokia Corporation)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Hansen, Mark H., Shriver, Elizabeth A.
• Granted Patent: US 7,194,454 B2
• Time in Patent Office: Days
• US Class Current: 707/3
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9532 Query formulation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A method for organizing records of a database by topical relevance generates statistics on relevance by monitoring search terms used and search paths traversed by a database user community. Records reviewed most often in relation to a given search term are assumed to be most relevant to that search term in the eyes of members of the user community. Additionally, a record reviewed in relation to a plurality of search terms is determined to be related by topical relevance to other records reviewed in relation to that plurality of search terms. Again, a probability is calculated, based on a frequency of record review and search terms used, as a measure of this record topical relevance. An embodiment directed toward Internet searches provides for seeding the probability calculations with information from labeled data available from open source Internet directories. The activities of the user community are monitored, for example, at a proxy server, or by reviewing proxy server logs. Other monitoring points are contemplated.

Claims

1. A method of improving search of a database, the method comprising:

monitoring user search activity in a user population;

extracting search sessions, defined by search queries and paths, from user search activity;

determining groups of semantically related queries or paths based on search session data;

determining probabilities that records in the database are relevant for each query or path group;

maintaining a table associating an index for each record in the database with the probability that the record is relevant for each query or path group; and

,

supplementing search results with information regarding records from the database with tabulated relevance probabilities.

2. The method of improving database search of claim 1 wherein the search is web page search and the database includes a collection of available web pages.

3. The method of improving database search of claim 1 wherein the search is web page search and the database includes a collection of publicly available Internet web pages.

4. The method of improving database search of claim 1 wherein the search is web page search and the database includes a collection of private intranet web pages.

5. The method of improving database search of claim 2 wherein monitoring user search activity in a user population and extracting search sessions from user search activity includes off-line processing of proxy server access logs to determine search sessions.

6. The method of improving database search of claim 2 wherein monitoring user search activity in a user population and extracting search sessions from user search activity includes on-line processing in a proxy server to determine search sessions.

7. The method of improving database search of claim 2 wherein monitoring user search activity in a user population and extracting search sessions from user search activity includes off-line processing of proxy server access logs to determine complete search sessions.

8. The method of improving database search of claim 2 wherein monitoring user search activity in a user population and extracting search sessions from user search activity includes on-line processing in a proxy server to determine complete search sessions.

9. The method of improving database search of claim 1 includes extracting topic sessions, defined by multiple search sessions where the queries include refinements, from user search activity.

10. The method of improving database search of claim 1 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes clustering queries based on a similarity of the associated search paths using a Poisson mixture model.

11. The method of improving database search of claim 1 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes using predetermined labeled data.

12. The method of improving database search of claim 11 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes applying an approximate Expectation-Maximization algorithm to the predetermined labeled data.

13. The method of improving database search of claim 11 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes using predetermined labeled data by seeding query or path groups.

14. The method of improving database search of claim 1 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes clustering queries or paths in an on-line fashion.

15. The method of improving database search of claim 1 wherein maintaining a table associating the index for each record includes using a database to store the table.

16. The method of improving database search of claim 1 wherein supplementing search results with information regarding records from the database with tabulated relevance probabilities includes displaying the information in a separate area of the display from results of a search engine.

17. The method of improving database search of claim 1 wherein supplementing search results with information regarding records from the database with tabulated relevance probabilities includes modifying the order of the information.

18. The method of improving database search of claim 1 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes clustering data.

19. The method of improving database search of claim 1 wherein determining groups of semantically related queries based on search session data and determining probabilities that records in the database are relevant for each query group includes clustering queries based on a similarity of items in their associated search paths.

20. The method of improving database search of claim 19 wherein determining groups of semantically related queries or paths based on search session data and determining probabilities that records in the database are relevant for each query or path group includes clustering queries or paths using an Expectation-Maximization algorithm.

21. A method of improving search of a database, the method comprising:

monitoring user search activity in a user population;

extracting search sessions, defined by search queries and paths, from user search activity;

determining groups of semantically related paths based on search session data;

determining probabilities that records in the database are relevant for each path group;

maintaining a table associating an index for each record in the database with the probability that the record is relevant for each path group; and

,

supplementing search results with information regarding records from the database with tabulated relevance probabilities.

22. A method of improving search of a database, the method comprising:

monitoring user search activity in a user population;

extracting search sessions, defined by search queries and paths, from user search activity;

determining groups of semantically related queries based on search session data;

determining probabilities that records in the database are relevant for each query group;

maintaining a table associating an index for each record in the database with the probability that the record is relevant for each query group; and

,

supplementing search results with information regarding records from the database with tabulated relevance probabilities.