AUTOMATIC DIAGNOSIS OF SEARCH RELEVANCE FAILURES

• US 20100121841A1
• Filed: 11/13/2008
• Published: 05/13/2010
• Est. Priority Date: 11/13/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wang, Yi-Min, Yin, Xiaoxin, He, Li-wei, Fan, Jinliang, Tan, Wenzhao
• Granted Patent: US 8,041,710 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Search relevance failures are diagnosed automatically. Users presented with unsatisfactory search results can report their dissatisfaction through various mechanisms. Dissatisfaction reports can trigger automatic investigation into the root cause of such dissatisfaction. Based on the identified root cause, a search engine can be modified to resolve the issue creating dissatisfaction thereby improving search engine quality.

Claims

1. A system for diagnosis of search relevance failures, comprising:

a collection component that acquires search result dissatisfaction reports; and

a diagnostic component that automatically identifies one or more potential causes of reported dissatisfaction to facilitate resolution thereof.

2. The system of claim 1, the diagnostic component employs a decision diagram to automate identification of the causes.

3. The system of claim 1, the diagnostic component employs black-box techniques to identify the one or more potential causes.

4. The system of claim 1, further comprising a component that classifies causes in accordance with a number of predetermined classes or categories.

5. The system of claim 4, the categories address one of crawler, index, document candidate set, ranking, or diversity constraint issues.

6. The system of claim 1, the dissatisfaction reports identify relevant URLs that are missing from search results.

7. The system of claim 1, the dissatisfaction reports identify duplicate URLs in search results.

8. The system of claim 1, the collection component acquires automatically generated dissatisfaction reports.

9. The system of claim 1, further comprising a data store that houses the dissatisfaction reports to serve as test suites.

10. The system of claim 1, further comprising a correction component that automatically augments a search engine in accordance with the one or more identified causes to address the dissatisfaction report.

11. A search engine diagnostic method, comprising:

receiving a dissatisfaction report identifying a missing relevant uniform resource locator (URL) in search results in response to query; and

diagnosing root cause of the missing URL automatically.

12. The method of claim 11, further comprising identifying a class of specified causes to which the root cause belongs, and forwarding the dissatisfaction report and the root cause to an entity associated with the identified class for resolution.

13. The method of claim 11, comprising verifying that the missing URL is properly indexed.

14. The method of claim 13, comprising determining whether the URL is in a candidate set for the query.

15. The method of claim 14, comprising determining whether an alteration of the query results in presentation of the URL.

16. The method of claim 15, further comprising determining whether a core ranking issue exists.

17. The method of claim 16, further comprising determining whether the URL is being excluded as a result of a host-based diversity constraint.

18. A method for improving search engine quality, comprising:

receiving a dissatisfaction report that identifies a missing relevant URL for a search query;

diagnosing automatically a root cause that prevented presentation of the URL in search results;

categorizing the potential cause; and

initiating search engine modification to address the missing URL as a function of the categorized cause.

19. The method of claim 18, further comprising extracting the dissatisfaction report from an unstructured format.

20. The method of claim 18, comprising notifying a person or team associated with the categorized cause of the dissatisfaction report.