Evaluating semantic interpretations of a search query

• US 10,353,964 B2
• Filed: 03/11/2015
• Issued: 07/16/2019
• Est. Priority Date: 09/15/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Venugopal, Ashish, Uszkoreit, Jakob D., Blitzer, John, Anderson, Edward Everett
• Primary Examiner(s): Allen, Brittany N
• Assistant Examiner(s): Nafziger, David M
• Application Number:
  US14/644,803
• Publication Number:
  US 20160078131A1
• Time in Patent Office: 1,588 Days
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

The present disclosure relates to evaluating different semantic interpretations of a search query. One example method includes obtaining a set of search results for a particular search query submitted to a search engine; obtaining a set of semantic interpretations for the particular search query; obtaining, for each semantic interpretation of the set, a canonical search query; generating a modified search query based at least in part on the particular search query and the canonical search query for the semantic interpretation; obtaining a set of search results for the modified search query for the semantic interpretation; and determining, for each semantic interpretation of the set, a degree of similarity between (i) the set of search results of the modified search query for the semantic interpretation, and (ii) the set of search results for the particular search query.

Claims

1. A computer-implemented method executed by one or more processors, the method comprising:

obtaining a set of search results for a particular search query submitted to a search engine, wherein the particular search query includes a substring that is a subset of terms of the query, and where the subset of terms identifies a particular entity;

obtaining a set of semantic interpretations for the particular search query, each semantic interpretation representing a candidate intent associated with the particular entity identified by the subset of terms included in the particular search query;

obtaining, for each semantic interpretation of the set, a canonical search query;

generating, for each semantic interpretation of the set, a modified search query based at least in part on the particular search query and the canonical search query for the semantic interpretation, wherein generating the modified search query for each semantic interpretation includes replacing, in the particular search query, the substring included in the particular search query identifying the particular entity with an alternate substring that includes (i) the subset of terms that identify the particular entity included in the particular search query and (ii) includes a second set of terms from the canonical search query for the semantic interpretation, wherein the second set of terms were not included in the particular search query;

obtaining, for each semantic interpretation of the set, a set of search results for the modified search query for the semantic interpretation;

determining, for each semantic interpretation of the set, a degree of similarity between (i) the set of search results of the modified search query for the semantic interpretation, and (ii) the set of search results for the particular search query, wherein determining the degree of similarity is based at least in part on comparing attributes of the set of search results for the modified search query with attributes of the set of search results for the particular search query; and

selecting a particular semantic interpretation of the set of semantic interpretations based on the degrees of similarity between the set of search results of the modified search query for the particular semantic interpretation and the set of search results for the particular search query.

2. The method of claim 1, wherein generating the modified search query for each semantic interpretation includes reformatting the particular search query to match the canonical search query for the semantic interpretation.

3. The method of claim 1, wherein determining the degree of similarity is based at least in part on the size of an intersection between the set of search results for the modified search query and the set of search results for the particular search query.

4. The method of claim 1, wherein determining the degree of similarity is based at least in part on the size of a difference between the set of search results for the modified search query and the set of search results for the particular search query.

5. The method of claim 1, wherein determining the degree of similarity is based at least in part on the frequency of occurrence of particular keywords associated with the particular search query in the set of search results for the modified search query and the set of search results for the particular search query.

6. The method of claim 1, wherein determining the degree of similarity is based at least in part on comparing an order of the set of search results for the modified search query with an order of the set of search results for the particular search query.

7. A non-transitory, computer-readable medium storing instructions operable when executed to cause at least one processor to perform operations comprising:

obtaining a set of search results for a particular search query submitted to a search engine, wherein the particular search query includes a substring that is a subset of terms of the query, and where the subset of terms identifies a particular entity;

obtaining a set of semantic interpretations for the particular search query, each semantic interpretation representing a candidate intent associated with the particular entity identified by the subset of terms included in the particular search query;

obtaining, for each semantic interpretation of the set, a canonical search query;

generating, for each semantic interpretation of the set, a modified search query based at least in part on the particular search query and the canonical search query for the semantic interpretation, wherein generating the modified search query for each semantic interpretation includes replacing, in the particular search query, the substring included in the particular search query identifying the particular entity with an alternate substring that includes (i) the subset of terms that identify the particular entity included in the particular search query and (ii) includes a second set of terms from the canonical search query for the semantic interpretation, wherein the second set of terms were not included in the particular search query;

obtaining, for each semantic interpretation of the set, a set of search results for the modified search query for the semantic interpretation;

determining, for each semantic interpretation of the set, a degree of similarity between (i) the set of search results of the modified search query for the semantic interpretation, and (ii) the set of search results for the particular search query, wherein determining the degree of similarity is based at least in part on comparing attributes of the set of search results for the modified search query with attributes of the set of search results for the particular search query; and

selecting a particular semantic interpretation of the set of semantic interpretations based on the degrees of similarity between the set of search results of the modified search query for the particular semantic interpretation and the set of search results for the particular search query.

8. The computer-readable medium of claim 7, wherein generating the modified search query for each semantic interpretation includes reformatting the particular search query to match the canonical search query for the semantic interpretation.

9. The computer-readable medium of claim 7, wherein determining the degree of similarity is based at least in part on the size of an intersection between the set of search results for the modified search query and the set of search results for the particular search query.

10. The computer-readable medium of claim 7, wherein determining the degree of similarity is based at least in part on the size of a difference between the set of search results for the modified search query and the set of search results for the particular search query.

11. The computer-readable medium of claim 7, wherein determining the degree of similarity is based at least in part on the frequency of occurrence of particular keywords associated with the particular search query in the set of search results for the modified search query and the set of search results for the particular search query.

12. The computer-readable medium of claim 7, wherein determining the degree of similarity is based at least in part on comparing an order of the set of search results for the modified search query with an order of the set of search results for the particular search query.

13. A system comprising:

memory for storing data; and

one or more processors configured to perform operations comprising;

obtaining a set of search results for a particular search query submitted to a search engine, wherein the particular search query includes a substring that is a subset of terms of the query, and where the subset of terms identifies a particular entity;

obtaining a set of semantic interpretations for the particular search query, each semantic interpretation representing a candidate intent associated with the particular entity identified by the subset of terms included in the particular search query;

obtaining, for each semantic interpretation of the set, a canonical search query;

generating, for each semantic interpretation of the set, a modified search query based at least in part on the particular search query and the canonical search query for the semantic interpretation, wherein generating the modified search query for each semantic interpretation includes replacing, in the particular search query, the substring included in the particular search query identifying the particular entity with an alternate substring that includes (i) the subset of terms that identify the particular entity included in the particular search query and (ii) includes a second set of terms from the canonical search query for the semantic interpretation, wherein the second set of terms were not included in the particular search query;

obtaining, for each semantic interpretation of the set, a set of search results for the modified search query for the semantic interpretation;

determining, for each semantic interpretation of the set, a degree of similarity between (i) the set of search results of the modified search query for the semantic interpretation, and (ii) the set of search results for the particular search query, wherein determining the degree of similarity is based at least in part on comparing attributes of the set of search results for the modified search query with attributes of the set of search results for the particular search query; and

selecting a particular semantic interpretation of the set of semantic interpretations based on the degrees of similarity between the set of search results of the modified search query for the particular semantic interpretation and the set of search results for the particular search query.

14. The system of claim 13, wherein generating the modified search query for each semantic interpretation includes reformatting the particular search query to match the canonical search query for the semantic interpretation.