System and process for automatically explaining probabilistic predictions

• US 6,831,663 B2
• Filed: 05/24/2001
• Issued: 12/14/2004
• Est. Priority Date: 05/24/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rounthwaite, Robert, Chickering, David Maxwell, Heckerman, David E.
• Primary Examiner(s): Kincaid, Kristine
• Assistant Examiner(s): PESIN, BORIS M
• Application Number:
  US09/681,709
• Publication Number:
  US 20020180805A1
• Time in Patent Office: 1,300 Days
• Field of Search: 349/700, 395/50, 395/52, 395/63, 395/64, 705/1, 705/5, 705/10, 705/14, 705/26, 705/27, 707/2, 707/3, 707/4, 707/102, 707/104.1
• US Class Current: 715/700
• CPC Class Codes: G06Q 10/02 Reservations, e.g. for tickets, services or events
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

The system and method of the present invention automatically assigns “scores” to the predictor/variable value pairs of a conventional probabilistic model to measure the relative impact or influence of particular elements of a set of topics, items, products, etc. in making specific predictions using the probabilistic model. In particular, these scores measure the relative impact, either positive or negative, that the value of each individual predictor variable has on the posterior distribution of the target topic, item, product, etc., for which a probability is being determined. These scores are useful for understanding why each prediction is make, and how much impact each predictor has on the prediction. Consequently, such scores are useful for explaining why a particular prediction or recommendation was made.

Claims

1. A computer-readable medium having computer executable instructions for determining an influence of at least one possible user choice on at least one prediction of likely user choices, said computer executable instructions comprising:

predicting at least one likely user choice based on a probabilistic model and a set of preferences for the user;

determining the influence of at least one of the possible user choices on the prediction of at least one likely user choice by calculating probabilistic explanation scores for each of at least one possible user choice, said explanation scores representing a probabilistic impact of predictor/variable value pairs on a posterior distribution corresponding to the prediction of each likely user choice;

sorting the explanation scores; and

providing one or more of the sorted explanation scores via a user interface.

2. The computer-readable medium of claim 1 wherein the set of user preferences is determined implicitly.

3. The computer-readable medium of claim 1 wherein the set of user preferences is determined explicitly.

4. The computer-readable medium of claim 1 wherein sorting the explanation scores comprises sorting each explanation score in order from highest to lowest.

5. The computer-readable medium of claim 4 wherein a predetermined number of highest value explanation scores from the sorted explanation scores are provided via a user interface.

6. The computer-readable medium of claim 1 wherein one or more highest value explanation scores from the sorted explanation scores are automatically provided as part a natural language explanation for explaining each prediction of likely user choices.

7. The computer-readable medium of claim 4 wherein a predetermined number of lowest value explanation scores from the sorted explanation scores are provided via a user interface.

8. The computer-readable medium of claim 4 wherein one or more of the sorted explanation scores are provided via a user interface as part a natural language explanation for explaining each prediction of likely user choices.

9. A system for automatically assigning scores to members of a set of at least one predictor/variable value pair representing likely user choices, comprising:

obtaining a set of user preferences for a particular user;

computing predictor/variable value pairs for each possible user choice using the set of user preferences in combination with a probabilistic model;

computing at least one prediction of likely user choices based upon the predictor/variable value pairs;

calculating explanation scores for at least one of the predictor/variable value pairs for each prediction;

determining the influence of at least one member of the set of predictor/variable value pairs for each prediction based upon the scores calculated for the predictor/variable value pairs; and

wherein at least one explanation score is provided via a user interface as part of natural language explanation for explaining why each prediction of likely user choices was made.

10. The system of claim 9 wherein the set of user preferences is obtained by examining a set of responses to a set of questions provided to the particular user.

11. The system of claim 9 wherein the set of user preferences is obtained by automatically examining at least one pattern of behavior of the particular user.

12. The system of claim 9 wherein the set of user preferences is obtained automatically by using a combination of implicit and explicit methods for determining the preferences of the particular user.

13. The system of claim 9 wherein the probabilistic model is automatically generated by examining a behavior of a group of at least one user.

14. The system of claim 9 wherein the probabilistic model is automatically generated using machine learning techniques.

15. A method for identifying positive and negative influences on recommendations of likely user choices, comprising:

automatically computing predictor/variable value pairs for each possible user choice using a set of user preferences in combination with a probabilistic model;

automatically providing at least one recommendation of likely user choices based upon the predictor/variable value pairs;

automatically calculating an influence of each possible user choice for each recommendation based upon scores calculated for the predictor/variable value pairs, said scores representing a probabilistic impact of each possible user choice on a posterior distribution representing a probability of each recommendation; and

wherein the influence of at least one possible user choice on the at least one recommendation of likely user choices is provided in a human readable format via a user interface.

16. The method of claim 15 wherein the set of user preferences is provided by user input via a user interface.

17. The method of claim 15 wherein the set of user preferences is provided by examining Internet web browsing behavior of a user.

18. The method of claim 15 wherein the probabilistic model is automatically learned using a set of behavioral data from a set of at least one user.

19. The method of claim 18 wherein the probabilistic model is automatically learned using machine learning techniques.

20. The method of claim 15 wherein providing the influence of at least one possible user choice in a human readable format comprises providing the influence as a natural language explanation via a user interface.