Method and apparatus for adapting a search classifier based on user queries

• US 20040111419A1
• Filed: 12/05/2002
• Published: 06/10/2004
• Est. Priority Date: 12/05/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Seiber, Scott E., Cook, Daniel B., Goldberg, Matthew A., Oftedal, Chad S.
• Granted Patent: US 7,266,559 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A method and computer-readable medium are provided for constructing a classifier for classifying search queries. The classifier is constructed by receiving a query from a user and applying the query to a classifier to identify the task. An unsupervised mapping between the query and the task is then identified and is used to train a new classifier. Under one embodiment, the unsupervised mapping is identified based on a user'"'"'s selection of the task.

Claims

1. A method of constructing a classifier for classifying user inputs, the method comprising:

receiving an input from a user;

applying the input to a first classifier to identify a category;

creating an unsupervised mapping between the input and the category identified by the first classifier; and

using the unsupervised mapping to train a new classifier.

2. The method of claim 1 wherein the input is a query and the category is a task.

3. The method of claim 2 further comprising receiving an indication that the user has selected the identified task before creating the unsupervised mapping.

4. The method of claim 1 wherein creating an unsupervised mapping comprises determining a confidence level for the unsupervised mapping.

5. The method of claim 4 wherein using the unsupervised mapping comprises determining that the confidence level for the unsupervised mapping is within a range of desired confidence levels and using the unsupervised mapping based on that determination.

6. The method of claim 1 further comprising testing the performance of the new classifier and replacing the first classifier with the new classifier if the performance of the new classifier is better than the performance of the first classifier.

7. The method of claim 1 further comprising using the unsupervised mapping to train a second new classifier.

8. The method of claim 6 further comprising testing the new classifier and the second new classifier to determine which is a better performing new classifier.

9. The method of claim 8 further comprising replacing the first classifier with the better performing new classifier.

10. The method of claim 9 wherein replacing the first classifier comprises only replacing the first classifier if the better performing new classifier performs better than the first classifier.

11. The method of claim 1 further comprising:

receiving a plurality of user inputs;

applying each input to the first classifier to identify respective category;

creating separate unsupervised mappings for each input and each respective category; and

selecting fewer than all of the unsupervised mappings to train the new classifier.

12. The method of claim 11 further comprising:

determining a confidence value for each unsupervised mapping; and

selecting the unsupervised mappings used to train the new classifier based on the confidence values.

13. The method of claim 12 further comprising:

receiving a set of desired confidence value ranges; and

selecting those unsupervised mappings with confidence values in the desired confidence value ranges.

14. The method of claim 11 wherein selecting fewer than all of the unsupervised mappings comprises selecting the mappings based on when they were produced.

15. A computer-readable medium having computer-executable instructions for performing steps comprising:

applying a user query to an automated classifier to identify a task;

providing the task to the user;

receiving an indication that the user selected the task; and

using the query and the task to train a new classifier based in part on the user'"'"'s selection of the task.

16. The computer-readable medium of claim 15 further comprising testing the performance of the new classifier and determining whether to replace the automated classifier with the new classifier based on the testing.

17. The computer-readable medium of claim 15 further comprising:

identifying a first task and a second task from the user query;

providing the first task and the second task to the user;

receiving an indication that the user selected the first task;

receiving an indication that the user selected the second task; and

using a mapping between the query and the first task and a mapping between the query and the second task to train the new classifier.

18. The computer-readable medium of claim 15 further comprising:

applying multiple different user queries to the automated classifier to identify multiple tasks; and

for each query;

providing a task identified for the query to a user; and

receiving an indication that the user has selected the task.

19. The computer-readable medium of claim 18 further comprising generating a mapping between each query and each selected task.

20. The computer-readable medium of claim 19 further comprising selecting fewer than all of the mappings to train the new classifier.

21. The computer-readable medium of claim 20 further comprising using a first set of mappings to train a first new classifier and a second set of mappings, different from the first set of mappings, to train a second new classifier.

22. The computer-readable medium of claim 21 further comprising testing the first new classifier and the second new classifier to determine which performs better.

23. The computer-readable medium of claim 20 wherein selecting fewer than all of the mappings to train the new classifier comprises selecting mappings based on when the mappings were generated.

24. The computer-readable medium of claim 23 wherein training a classifier comprises training each task separately.

25. The computer-readable medium of claim 24 wherein a first task is trained by selecting a first percentage of mappings produced for the first task, and a second task is trained by selecting a second percentage of mappings produced for the second task, the first percentage being different from the second percentage.