Relevance-weighted navigation in information access, search and retrieval

• US 20080189269A1
• Filed: 11/06/2007
• Published: 08/07/2008
• Est. Priority Date: 11/07/2006
• Status: Active Grant

• Current Assignee: Microsoft Corporation
• Original Assignee: Fast Search & Transfer ASA (Microsoft Corporation)
• Inventors: Olsen, Oystein Haug
• Granted Patent: US 7,966,305 B2
• Time in Patent Office: Days
• US Class Current: 707/5
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

In a method for information access, search, and retrieval over a data communication system generally, wherein a query is applied to a set of documents, a result set of the matching documents and query-dependent subsections of the matching documents are identified. The method comprises computing summary information on document scopes, values and associated weights across the result sets and weighting the values with a summary value metric which provides a measure of the probability of a value. In a search engine (100) capable of supporting and implementing the above method, the search engine comprises as per se known subsystems for performing search and retrieval in the form of one or more core search engines (101), a content application programming interface (102), a content analysis stage (103) and a client application programming interface (106) connected to the core search engine (101) via query analysis and result analysis stages (105;106). In addition the search engine (100) for supporting the above method comprises a first module (108) for specifying summary value metrics (SVMs), second and third modules (109;110) for creating scopes and for assigning summary value metrics (SVMs) to the created scopes, and a fourth module (111) for indexing scopes, values and SVMs.

Claims

1. A method for computing summary information on documents containing hierarchical named scopes comprising a plurality of associations between a scope and a value or between a scope and a value-weight pair, wherein the method comprises steps for

applying a query to a set of documents, and identifying a result set of matching documents and query-dependent subsections of the matching documents,

and wherein the method is

characterized by an additional step for computing summary information on document scopes, their values, and associated weights across the result set, and

weighting the values with a summary value metric (SVM), said summary value metric being a combination of weighted statistical and dynamic properties of an attribute value, thus providing a measure of the probability of the attribute value.

2. A method according to claim 1,

characterized by said documents having an implicit and anonymous root element such that the said documents effectively have a set of named elements and attributes, i.e. a one-level structure.

3. A method according to claim 1,

characterized by performing an indexing time analysis of the said set of documents to discover and encode semantic structures of the documents, and assigning weights to the encoded structures.

4. A method according to claim 1,

characterized by scope values comprising weighted associations to an ontology structure.

5. A method according to claim 4,

wherein said ontology structure is a taxonomy, and each value-weight pair gives the probability of belonging to a taxonomy node representing the scope values.

6. A method according to claim 4,

characterized by mapping the scope values through the relations of an ontology structure subject to weight modifications associated with said relations in the ontology structure

7. A method according to claim 1,

characterized by computing the summary information with supervised or unsupervised clustering of selected document scopes, taking weights associated with the scope values into account, said summary information responding to the probability of cluster membership.

8. A method according to claim 1,

characterized by the query comprising fragments that are specifically aimed at the SVM and an aggregation of SVMs.

9. A method according to claim 1,

characterized by the SVM responding to at least one of a relevance score within a query context (query fragment) and specifically a final relevance score of the documents in the result set;

a presence or absence of named document scopes, or the values of named document scopes and associated weight of these values;

if any.

10. A method according to claim 9, wherein the SVM responds to the relevance score within a query context,

characterized by the SVM further responding to one or more of a token/word positional proximity in the document between predicates in the query;

a character positional proximity in the document between predicates in the query;

a number of matches in the document, as selected by a subquery, between predicates in the query;

an aggregation on the matches in the document, as selected by a subquery, between predicates in the query;

a semantic distance of tokens/words in an ontology;

numeric values specified in the query and numeric values of scopes as selected by a sub-query, said numeric values include integers, real numbers, time, and geographical position; and

a distance measure between a document vector specified in or implied by the query and a vector value being selected from the matching documents.

11. A method according to claim 1,

characterized by comprising the creation of a contextual navigation by associating query refinements to the summary information, said query refinement including result set filtering and relevance boosting.

12. A method according to claim 1,

characterized by the summary information being weighted frequency information.

13. A method according to claim 12,

characterized by defining query predicates, a relevance calculation, an aggregation, or query refinements at least partly in terms of one of Java™

source code or byte code, XPatch or XQuery (including XQuery full-text), or ontology languages.

14. A method according to claim 1,

characterized by identifying which document scopes are related to the query, and identifying dominant values for said identified scopes, whereby relevant facts and knowledge are enhanced

15. A method according to claim 14,

characterized by presenting the identified scopes and the dominant values.

16. A method according to claim 1,

characterized by listing the most relevant scope and value associations are listed in an object presenting triples of scope value and weighted frequency ranked by decreasing weighted frequency across a set of named scopes.

17. A method according to claim 1,

characterized by subjecting the computation of the summary information to an error constraint, allowing the process to ignore some values in an early stage thereof

18. A method according to claim 1,

characterized by computing the summary information of the highest ranked documents in the result set, and optionally estimating an error in the summary information due to ignoring the other documents in the result set and returned in an error component of a summary object.

19. A method according to claim 17 or claim 18,

characterized by distributing the computation of the summary information over multiple processing units.

20. A method according to claim 1,

characterized by including co-occurrence statistics in the computation of summary information.

21. A method according to claim 20,

characterized by comprising presenting the co-occurrence statistics between two scopes as a two-dimensional heat map highlighting most relevant co-occurrence associations, and/or said co-occurrence statistics as a navigator on most relevant co-occurrence associations.

22. A method according to claim 1,

characterized by determining a popularity score to SVM, said popularity score being determined as a result of an analysis of content references such as hyperlinks and citations, and document actions such as views and purchases.

23. A method according to claim 1,

characterized by determining said values and SVM by an analysis of document actions in response to a result set of query, including one or more of

presentation of documents in the result set;

user views of documents in the result set;

purchases of objects represented by documents in the result set;

recommending objects represented by documents in the result set;

reviewing object represented by documents in the result set;

presentation of navigator entries in the result set; and

user selection and query refinement of navigator entries.

24. A search engine (100) capable of supporting and implementing the method according to any of the preceding claims in information access or search systems, wherein the search engine (100) is applied to accessing, searching, retrieving and analyzing information from content repositories available over data communication networks, including extranets and intranets, and presenting search and analysis results for end users, wherein the search engine comprises at least a core search engine (101), a content application programming interface (102) (content API) connected to the at least one core search engine (101) via content analysis stage (103), and a query application programming interface (107) connected to said at least one core search engine (101) via respective query analysis and result analysis stages (105;

106), and wherein the search engine (100) is

characterized in

comprising a first module (108) for specifying summary value metrics (SVMs) to hierarchical scopes, said first module (108) being included either in the content API (102) or in the content analysis stage (103), a second module (109) for scope creation from recognizing explicit format structures in the content, a third module (110) for scope creation from analysis of the content, said second and third modules (109;

110) being included in the content analysis stage (103) and adapted for assigning summary value metrics (SVMS) to said created scopes, and a fourth module (111) for indexing scopes, values and SVMs, said fourth module being included in an indexing subsystem (103a) of the core search engine (103).