Automatic diagnosis of search relevance failures
First Claim
Patent Images
1. A system for diagnosis of search relevance failures, the system comprising:
- memory;
at least one processor communicatively coupled to the memory;
a dissatisfaction report generator component to automatically generate search result dissatisfaction reports, the dissatisfaction report generator component including;
a human judgment component stored in the memory and executed by the at least one processor to;
determine a difference in relevance of search results provided by multiple search engines in response to an associated query; and
generate one or more search result dissatisfaction reports based in part on the difference; and
a click log component stored in the memory and executed by the at least one processor to;
analyze recorded clicks on search results in one or more query click logs;
determine if one or more portions of the search results are irrelevant to the associated query based in part on the analyzing of the recorded clicks; and
generate one or more search result dissatisfaction reports if one or more portions of the search results are determined to be irrelevant;
a collection component that acquires the search result dissatisfaction reports; and
a diagnostic component that automatically identifies one or more potential causes of reported dissatisfaction to facilitate resolution thereof.
2 Assignments
0 Petitions
Accused Products
Abstract
Search relevance failures are diagnosed automatically. Users presented with unsatisfactory search results can report their dissatisfaction through various mechanisms. Dissatisfaction reports can trigger automatic investigation into the root cause of such dissatisfaction. Based on the identified root cause, a search engine can be modified to resolve the issue creating dissatisfaction thereby improving search engine quality.
-
Citations
19 Claims
-
1. A system for diagnosis of search relevance failures, the system comprising:
-
memory; at least one processor communicatively coupled to the memory; a dissatisfaction report generator component to automatically generate search result dissatisfaction reports, the dissatisfaction report generator component including; a human judgment component stored in the memory and executed by the at least one processor to; determine a difference in relevance of search results provided by multiple search engines in response to an associated query; and generate one or more search result dissatisfaction reports based in part on the difference; and a click log component stored in the memory and executed by the at least one processor to; analyze recorded clicks on search results in one or more query click logs; determine if one or more portions of the search results are irrelevant to the associated query based in part on the analyzing of the recorded clicks; and generate one or more search result dissatisfaction reports if one or more portions of the search results are determined to be irrelevant; a collection component that acquires the search result dissatisfaction reports; and a diagnostic component that automatically identifies one or more potential causes of reported dissatisfaction to facilitate resolution thereof. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A search engine diagnostic method implemented by a computer, the method comprising:
-
receiving a dissatisfaction report identifying a missing uniform resource locator (URL) in search results in response to a query, the missing URL being relevant to the query; and diagnosing root cause of the missing URL automatically by the computer by performing actions including; determining whether the missing URL is in a candidate set for the query by determining keywords of the query found at a document referenced by the missing URL; determining that one or more portions of a document referenced by the missing URL are not properly indexed; determining that at least one of the one or more portions of the document are relevant to the query; categorizing the root cause; and initiating search engine modification to address the missing URL as a function of the categorized cause. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. A method implemented by a computer for improving search engine quality, the method comprising:
-
receiving a dissatisfaction report that identifies a missing URL for a search query, the missing URL being relevant to the search query; diagnosing automatically by the computer a root cause that prevented presentation of the missing URL in search results by performing actions including; determining whether the missing URL is in a candidate set for the query based in part on determining keywords of the query found at a document referenced by the missing URL; determining whether an alteration of the query results in presentation of the missing URL in corresponding search results; and determining whether a portion of the document referenced by the missing URL is; improperly indexed; and relevant to the query; categorizing the root cause; and initiating search engine modification to address the missing URL as a function of the categorized cause. - View Dependent Claims (18, 19)
-
Specification