System and methods for inferring informational goals and preferred level of detail of results in response to questions posed to an automated information-retrieval or question-answering service

• US 7,519,529 B1
• Filed: 06/28/2002
• Issued: 04/14/2009
• Est. Priority Date: 06/29/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Horvitz, Eric J.
• Primary Examiner(s): ABEBE, DANIEL DEMELASH
• Application Number:
  US10/185,150
• Time in Patent Office: 2,482 Days
• Field of Search: 704/1, 704/7, 704/9, 369/30.04, 707/3, 707/4
• US Class Current: 704/7
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

A system and method for inferring informational goals and preferred level of details in answers in response to questions posed to computer-based information retrieval or question-answering systems is provided. The system includes a query subsystem that can receive an input query and extrinsic data associated with the query and which can output an answer to the query, and/or rephrased queries or sample queries. The query subsystem accesses an inference model to infer a probability distribution over a user'"'"'s goals, age, and preferred level of detail of an answer. One application of the system includes determining a user'"'"'s likely informational goals and then accessing a knowledge data store to retrieve responsive information. The system includes a natural language processor that parses queries into observable linguistic features and embedded semantic components that can be employed to retrieve the conditional probabilities from the inference model. The inference model is built by employing supervised learning and statistical analysis on a set of queries suitable to be presented to a question-answering system. Such a set of queries can be manipulated to produce different inference models based on demographic and/or localized linguistic data.

Claims

1. An automated information retrieval system, comprising:

A user interface for receiving a query;

an analyzer to process said query based upon at least one attribute related to a user, the analyzer infers one or more informational goals of the user from the query, the one or more informational goals of the user inferred by parsing the query into parts of speech and structural features and employing data obtained by parsing the query to access one or more decision trees in an inference model, the one or more decision trees store conditional probability distribution over the one or more informational goals given the parse data and a physical location of the user;

an answer generator that produces one or more responses to the query, the one or more responses are based, at least in part, on the one or more informational goals of the user inferred from the query, and the one or more responses vary in at least one of;

length, precision, and detail based on at least one of the inferred informational goals and the attribute; and

an inference clarifier that automatically clarifies an inference about at least one of the informational goals before producing the one or more responses by initiating a dialog with the user when the at least one of the inferred informational goals has a likelihood below a predefined probability threshold, wherein at least one of the user attribute or the inference model is refined upon an occurrence of the inference clarification.

2. The system of claim 1, the answer generator produces at least one of an answer responsive to a new query, a rephrased query, or a query that can be employed in query by example process.

3. The system of claim 1, the one or more responses are driven by at least one of a ranking process, an ordering process, a text formatting process, a diagramming process or a text focusing process.

4. The system of claim 1, the informational goals depend, at least in part, on at least one of, an age of a user, one or more relationships in which the user is engaged, or an application being employed by the user.

5. The system of claim 1, the at least one attribute describes characteristics of the user including at least one of age, ability, level of sophistication, or capability as possible indications of the user'"'"'s proclivity to process information.

6. The system of claim 1, the analyzer determines the level of detail of for an answer to the query based on at least one of an age of a user, the physical location of a user, one or more relationships in which the user is engaged, an application being employed by the user, or the at least one attribute.

7. The system of claim 1, the inferred informational goals can be employed to perform at least one of targeted advertising, link recommendations, age determinations, and demographic modeling.

8. The system of claim 1, where one or more inferred informational goals are employed to adapt at least one of a guided search process, a guided retrieval process, a post-search filtering process and a new text composition process.

9. The system of claim 1, where the inference clarifier further comprises a user interface to facilitate clarifying the inference associated with at least one of the informational goal, the at least one attribute, or the query.

10. The system of claim 1, the answer generator reformats the query from the user into a focused query directed to at least one search engine.

11. The system of claim 1, at least one of the answer generator and the analyzer are adapted to a search engine, the search engine is directed to retrieval activities associated with at least one of the informational goals or the attribute.

12. The system of claim 10, the reformatted query is employed to select a subset of search engines to process the query based in part on the informational goals and the attribute.

13. The system of claim 1, further comprising one or more processing components to process at least one of words, phrases, symbolic gestures, or speech from the user.

14. The system of claim 1, further comprising one or more formatting components to output at least one of words, phrases, symbolic gestures, or speech to the user.

15. The system of claim 1, the informational goals inferred from the query comprise information need, coverage wanted, coverage would give, topic and focus.

16. A method for automatically answering queries, comprising:

receiving a query from a user;

employing language processing to parse the query into component parts of speech and logical forms;

employing parse data produced by parsing the query to access a decision model, the decision model storing conditional probabilities associated with informational goals given the query within one or more decision trees;

inferring one or more informational goals from the decision model and a physical location of the user;

generating an answer related to the query and the one or more inferred informational goals;

automatically clarifying at least one inference associated with the one or more informational goals prior to generating the answer, via a dialog with the user when the inference has a probability below a predefined threshold; and

refining one or more of at least an attribute associated with the user or the decision model based upon occurrence of the automatic clarification of the at least one inference.

17. The method of claim 16, generating an answer related to the query and the one or more inferred informational goals comprises:

retrieving from a knowledge base a rephrased query, the query rephrasing based, at least in part, on the one or more inferred informational goals and one or more sample queries.

18. The method of claim 16, the inferred informational goals accurately reflect, at least seventy five percent of the time, actual informational goals of a consumer offering a question with less than seven phrases.

19. The method of claim 16, where the inferred informational goals reflect actual informational goals of a consumer offering a question based upon a probability threshold.

20. The method of claim 16, inferring the one or more informational goals further comprises considering at least one of an age of a user, one or more relationships in which the user is engaged or an application being employed by the user.

21. The method of claim 20, where inferring the one or more informational goals further comprises inferring one or more levels of detail for an answer to the query, the levels of detail determined in part on at least one attribute of the user.

22. The method of claim 21, further comprising performing automated analysis that includes at least one of determining sentence structure, sentence length, sentence phrasing, common word regularity, common word non-regularity, a document length, a document content, or a document type.

23. The method of claim 16, further comprising adapting one or more of a guided search process, a guided retrieval process, a post-search filtering process, or a new text composition process.

24. A computer readable medium storing computer executable instructions operable to perform a method for answering questions, the method comprising:

inputting a question;

employing language processing to parse the question into component parts of speech and logical forms;

employing parse data produced by parsing the question to access a decision model, the decision model storing conditional probabilities associated with informational goals given the query within one or more decision trees;

employing the decision model and a physical location of a user to infer one or more informational goals;

clarifying one or more of the inferences about the informational goals when at least one of the inferences has a likelihood below a probability threshold by conducting a dialog with the user;

determining that one or more of a user attribute or the decision model need refining based on occurrence of the dialog with the user; and

subsequently producing an answer related to the question and the one or more inferred informational goals.

25. A system for answering a question posed to an automated question answerer comprising:

means for initializing a model representing the likelihood that a certain type of answer is desired, the model being generated from probability distributions over a set of goals given a question and a physical location of a user, and stored in one or more data repositories;

means for parsing the question to identify parts of speech and structural features within the question, and employing the identified parts of speech and structural features to access one or more decision trees within the model to infer one or more of the goals;

means for automatically adapting the model over time;

means for determining one or more answers to the question based on likelihoods retrieved from accessing the model; and

means for conducting a dialog with a user to clarify one or more inferences made regarding the set of goals prior to determining the one or more answers when the likelihoods of at least one of the inferences is below a predefined thresholds wherein an initiation of the clarifying dialog determines that at least one of a user attribute or the model should be refined.

26. The system of claim 25, means for manually adapting the model.

27. A method to automatically clarify informational goals, comprising:

employing language processing to parse a query into observable linguistic features;

performing a probabilistic analysis on observable linguistic data received from parsing the query by assessing at least a conditional probability distribution over a set of one or more informational goals given the parse data and a physical location of a user, the conditional probability distribution stored in one or more decision trees, the one or more decision trees make up an inference model;

establishing a probability threshold related to an uncertainty in the analysis;

invoking a dialog with a user when the probabilistic analysis of at least one of the informational goals yields a determination below the probability threshold;

refining one or more of a user attribute or the decision model based on an occurrence of the dialog with the user; and

searching for information based upon an answer selected in response to the dialog.

28. The method of claim 27, modifying an information search further comprises:

analyzing the selected answer;

determining at least one attribute related to a user from at least one of the selected answer or an inference model; and

automatically generating a focused query to a search engine that is limited by the determined attribute.

29. The method of claim 28, further comprising:

receiving information items from the focused query;

determining a level of detail for the information items; and

presenting a subset of the information items to a user based upon matching the level of detail with at least one attribute associated with a user.

30. The method of claim 29, further comprising at least one of automatically filtering and pruning the information items before presentation to the user.

31. An automated information processing method, comprising:

processing inferences about a probability distribution over informational goals given a query, a physical location of a user, and parts of speech containing a focus of attention of the query in accordance with attributes of a user and most appropriate level of detail, wherein the probability distribution is stored in one or more data structures that are comprised in an inference model, wherein parsing the query into the parts of speech facilitates accessing the data structures in the inference model;

employing the inferences in at least one of a post-filter process, a reformulation process, a process for dynamically crafting an answer to the query; and

a process for driving a dialog in pursuit of refining the probability distribution, and a process for driving dialog in pursuit of a more appropriate query in order to satisfy the informational goals before crafting the answer when at least one of the inferences about the informational goals has a likelihood below a predefined probability threshold wherein one or more of at least an attribute associated with the user or the inference model is refined based upon occurrence of the dialog with the user.

32. The method of claim 31, the attributes relate to an age of the user and a level of detail suitable for the user, the attributes are inferred from evidential information drawn from words and linguistic structure of the query.

33. The method of claim 31, the attributes are inferred from at least one of a history of queries or profile information about the user.

34. The method of claim 31, the post filter process further comprising:

filtering a set of search results to automatically select results that are drawn from an appropriate genre of information.

35. The method of claim 31, the reformulation process further comprising at least one of:

reformulating a search query to a search engine;

adding terms and controls that identify the appropriate topic or genre of an article;

or

automatically adding terms in a search engine that are valuable for discovering a structure of an article.

36. The method of claim 35, further comprising at least one of:

analyzing at least one of topic terms, an abstract, or a reference to determine a genre of articles that have a standard structure of an abstract and a standard list of academic references at the end of the article;

or

determining more about topics that appear in children-oriented articles.

37. The method of claim 31, the process of dynamically crafting an answer to the query further comprises:

extracting at least one of an appropriate text, graphics, or media from one or more articles; and

using automated procedures for extracting information based on at least one of the informational goals, age, or focus of attention of the user.

38. The method of claim 31, the process of for driving a dialog in pursuit of refining the probability distribution further comprises:

driving the dialog over states of at least one of informational goals, age, appropriate level of detail, or focus of attention of the query.

39. The method of claim 31, the process of driving a dialog in pursuit of a more appropriate query further comprises at least one of:

seeking input from users on confirmation about the validity of reformulated queries, per the user'"'"'s informational goals;

performing automated procedures driven by formal decision-theoretic value-of-information methods;

or

performing heuristic procedures taking as inputs probability thresholds on such summaries as a maximum likelihood of states of interest and a distribution of probabilities over states of interest.

40. The method of claim 31, further comprising providing an autosummarizer for text to control a level of detail in an article from shorter versions to more verbose versions.

41. The method of claim 40, further comprising employing the level of detail and age to generate content that is provided in a manner that is elementary in the beginning but proceeds in a more sophisticated manner toward the end.

42. The method of claim 40, further comprising employing inferences about level of detail and age with articles that are encoded in a manner where concepts are represented hierarchically and respective levels of the hierarchy are marked by level of detail, wherein valid sequences can be constructed by walking the hierarchy in a breadth-first manner at different levels of the hierarchy, and further wherein the level of the hierarchy of concepts being employed is controlled by age and the level of detail.

43. The method of claim 42, further comprising determining the hierarchy via a Rhetorical Structure Theoretic encoding of content.