DYNAMIC METADATA FILTERING FOR CLASSIFIER PREDICTION

• US 20090024604A1
• Filed: 07/19/2007
• Published: 01/22/2009
• Est. Priority Date: 07/19/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: ZHAO, Lei, ZHAO, Yanbiao, CHEN, Zhanliang
• Granted Patent: US 7,925,645 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3346 using probabilistic model

Abstract

A classifier is used to predict relevant results with arbitrary filtering conditions specified by the user. The classifier model is stored as a database table and joined with a metadata properties table instead of calculating the query result probability using the full classifier model. A user-specified query based filter is applied to the joined tables to obtain the list of documents satisfying the filter. The probability is then computed using the sub-model.

Claims

1. A method to be executed at least in part in a computing device for predicting query results in a document search system utilizing metadata properties, the method comprising:

receiving a metadata property table;

receiving a classifier model table;

joining the metadata property table and the classifier model table;

applying a filter condition to the joined table based on user-specified query conditions;

obtaining a list of documents satisfying the filter condition;

computing a probability of a document satisfying a user-specified query based on the list of documents; and

presenting the probability to the user.

2. The method of claim 1, further comprising:

receiving a user-specified metadata schema; and

generating the metadata property table based on the received schema.

3. The method of claim 1, wherein the metadata property table includes a column for a property identifier, a column for a property name, a column for a property type, and a column for a property value.

4. The method of claim 1, wherein the classifier model table includes a column for a search term, a column for documents where each search term can be found, and a column for a frequency of occurrence of the search term.

5. The method of claim 4, wherein the classifier model table is derived from a sparse matrix that comprises occurrence data of each search term in each document stored by the document search system.

6. The method of claim 1, wherein the metadata properties include at least one of a single value property and a multi-value property.

7. The method of claim 1, wherein the same classifier model table is used without modification when the metadata property table is modified due to an update.

8. The method of claim 1, further comprising:

dynamically translating the filter condition to a Structured Query Language (SQL) where-clause for querying the classifier model table joined with the metadata property table.

9. The method of claim 1, wherein the classifier model is based on a Naï

ve Bayesian probability model.

10. The method of claim 10, wherein the probability is computed using:

$\log <br><br><br><br>\mathrm{Pr}<br><br>\left(\mathrm{query}|\mathrm{doc}\right)=\log <br><br><br><br>\mathrm{cache}<br><br>\left(\mathrm{doc}\right)+\underset{w_i\in <br><br><br><br>\mathrm{query}}{\sum <br><br>}<br><br><br><br>\mathrm{termprob}<br><br>\left(i,\mathrm{doc}\right)$ $\mathrm{termprob}<br><br>\left(i,\mathrm{doc}\right)=\log <br><br><br><br>\mathrm{Pr}<br><br>\left(w_i=1|\mathrm{doc}\right)-\log <br><br><br><br>\mathrm{Pr}<br><br>\left(w_i=0|\mathrm{doc}\right)$

where term probabilities (termprob) are derived from raw counts of a training data set associated with the classifier model.

11. A system for predicting query results in a document search system utilizing metadata properties, the system comprising:

a memory;

a processor coupled to the memory, capable of executing;

a service application configured to;

receive a user defined metadata schema;

generate a metadata property table based on the received schema;

a prediction engine configured to;

receive a classifier model table based on training data associated with documents stored by the document search system;

join the metadata property table and the classifier model table;

apply a filter condition to the joined table based on user-specified query conditions;

obtain a list of documents satisfying the filter condition; and

compute a probability of a document satisfying a user-specified query based on the list of documents.

12. The system of claim 11, wherein the service application is further configured to provide the probability to another application for presentation to a user requesting the query conditions.

13. Tie system of claim 11, further comprising:

a document store for storing documents searchable based on their metadata properties; and

a metadata store for storing metadata schema associated with the stored documents.

14. The system of claim 11, wherein the prediction engine is further configured to utilize a statistical algorithm that provides a monotone relationship between a probability of a document satisfying the user-specified query conditions based on a full classifier model and the probability of the document satisfying the user-specified query based on the list of documents.

15. The system of claim 14, wherein the statistical algorithm is based on a Naï

ve Bayesian model using a Bernoulli event model.

16. A computer-readable storage medium with instructions encoded thereon for predicting query results in a document search system utilizing metadata properties, the instructions comprising:

receiving custom metadata schema;

generating a metadata property table based on the received schema;

receiving a classifier model matrix based on a training data set associated with documents processed by the system;

generating a classifier model table from the sparse classifier model matrix;

joining the metadata property table and the classifier model table;

applying at least one filter condition to the joined table based on user-specified query conditions;

obtaining a list of documents satisfying the at least one filter condition; and

computing a probability of a document satisfying a user-specified query based on the list of documents.

17. The computer-readable storage medium of claim 16, wherein the instructions further comprise:

dynamically translating the at least one filter condition to an SQL where-clause for querying the classifier model table joined with the metadata properties table.

18. The computer-readable storage medium of claim 16, wherein the metadata property table includes a column for a property identifier, a column for a property name, a column for a property type, and a column for each property value;

and wherein the classifier model table includes a column for a search term, a column for documents where each search term can be found, and a column for a frequency of occurrence of the search term.

19. The computer-readable storage medium of claim 16, wherein the instructions further comprise:

enabling use of multi-value properties by generating row-based property tables that include a separate row for each value of a multi-value property and joining the row-based property tables with the classifier model table.

20. The computer-readable storage medium of claim 16, wherein the instructions further comprise:

enabling a query for performing the search to define a plurality of schemas to be included in the search.