Using web ranking to resolve anaphora

• US 10,083,226 B1
• Filed: 01/09/2017
• Issued: 09/25/2018
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Buchanan, Michael, Paskin, Mark Andrew, Gupta, Pravir Kumar
• Primary Examiner(s): Pham, Hung Q
• Application Number:
  US15/401,984
• Time in Patent Office: 624 Days
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/3325 Reformulation based on results of preceding query
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

The specification relates to a method of receiving a first query and a second query. The method analyzes the second query for a presence of anaphora. If anaphora is present, the method analyzes the first query for a presence of an entity that can be associated with the anaphora. If the analysis analyzing the first query returns two or more associated entities, the method forms a third query wherein the anaphora of the second query is replaced with one of the associated entities and forms a fourth query wherein the anaphora is replaced with the other of the associated entities. The third query and the fourth query are sent to a query-ranking engine. The third query and the fourth query receive a ranking and the higher-ranked query is sent to a search engine.

Claims

1. A method comprising:

receiving a first voice query;

receiving a second, follow-up voice query;

determining that an anaphora is present in the second, follow-up voice query by determining that the second, follow-up voice query includes a pronoun that refers to an entity that is not present in the second, follow-up voice query;

in response to determining that the anaphora is present in the second, follow-up voice query, determining that the first voice query is associated with a plurality of possible entities;

generating a plurality of candidate queries, wherein each candidate query of the plurality of candidate queries is generated by replacing the pronoun in the second, follow-up voice query with a corresponding possible entity;

determining that a corresponding possible entity in a first candidate query of the plurality of candidate queries is an entity associated with the anaphora from the second, follow-up voice query, wherein the entity associated with the anaphora from the second, follow-up voice query is the entity that is not present in the second, follow-up voice query and is referred to by the pronoun in the second, follow-up voice query; and

based on determining that the corresponding possible entity in the first candidate query is the entity associated with the anaphora, providing the first candidate query to a search engine and obtaining search results for the first candidate query from the search engine.

2. The method of claim 1, further comprising:

identifying an answer for the first candidate query in the search results for the first candidate query; and

audibly presenting the answer in response to the second, follow-up voice query.

3. The method of claim 1, wherein determining that the first voice query is associated with a plurality of possible entities further comprises the step of:

analyzing results of the first voice query for a presence of an entity associated with the anaphora.

4. The method of claim 1, wherein determining that a corresponding possible entity in a first candidate query of the plurality of candidate queries is an entity associated with the anaphora from the second, follow-up voice query comprises:

providing the plurality of candidate queries to a ranking engine;

obtaining a ranking of the candidate queries from the ranking engine; and

determining that the first candidate query is a highest-ranked candidate query according to the ranking.

5. The method of claim 4, wherein the ranking engine ranks the plurality of candidate queries using a data repository containing past query logs, and wherein the past query logs are associated with metadata describing one of a popularity of queries in the past query logs, a frequency of use of queries in the past voice query logs, and a commonality of queries in the past voice query logs.

6. The method of claim 4, further comprising:

modifying one or more of the plurality of candidate queries before sending the plurality of candidate queries to the ranking engine.

7. The method of claim 6, wherein modifying one or more of the plurality of candidate queries comprises one or more of dropping search terms, replacing search terms with synonyms, or introducing new terms.

8. The method of claim 4, wherein the ranking engine ranks the plurality of candidate queries by ranking n-grams contained within the candidate queries.

9. A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:

receiving a first voice query;

receiving a second, follow-up voice query;

determining that an anaphora is present in the second, follow-up voice query by determining that the second, follow-up voice query includes a pronoun that refers to an entity that is not present in the second, follow-up voice query;

in response to determining that the anaphora is present in the second, follow-up voice query, determining that the first voice query is associated with a plurality of possible entities;

generating a plurality of candidate queries, wherein each candidate query of the plurality of candidate queries is generated by replacing the pronoun in the second, follow-up voice query with a corresponding possible entity;

determining that a corresponding possible entity in a first candidate query of the plurality of candidate queries is an entity associated with the anaphora from the second, follow-up voice query, wherein the entity associated with the anaphora from the second, follow-up voice query is the entity that is not present in the second, follow-up voice query and is referred to by the pronoun in the second, follow-up voice query; and

based on determining that the corresponding possible entity in the first candidate query is the entity associated with the anaphora, providing the first candidate query to a search engine and obtaining search results for the first candidate query from the search engine.

10. The system of claim 9, further comprising:

identifying an answer for the first candidate query in the search results for the first candidate query; and

audibly presenting the answer in response to the second, follow-up voice query.

11. The system of claim 9, wherein determining that the first voice query is associated with a plurality of possible entities further comprises the step of:

analyzing results of the first voice query for a presence of an entity associated with the anaphora.

12. The system of claim 9, wherein determining that a corresponding possible entity in a first candidate query of the plurality of candidate queries is an entity associated with the anaphora from the second, follow-up voice query comprises:

providing the plurality of candidate queries to a ranking engine;

obtaining a ranking of the candidate queries from the ranking engine; and

determining that the first candidate query is a highest-ranked candidate query according to the ranking.

13. The system of claim 12, wherein the ranking engine ranks the plurality of candidate queries using a data repository containing past query logs, and wherein the past query logs are associated with metadata describing one of a popularity of queries in the past query logs, a frequency of use of queries in the past voice query logs, and a commonality of queries in the past voice query logs.

14. The system of claim 12, further comprising:

modifying one or more of the plurality of candidate queries before sending the plurality of candidate queries to the ranking engine.

15. The system of claim 14, wherein modifying one or more of the plurality of candidate queries comprises one or more of dropping search terms, replacing search terms with synonyms, or introducing new terms.

16. The system of claim 14, wherein the ranking engine ranks the plurality of candidate queries by ranking n-grams contained within the candidate queries.

17. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving a first voice query;

receiving a second, follow-up voice query;

determining that an anaphora is present in the second, follow-up voice query by determining that the second, follow-up voice query includes a pronoun that refers to an entity that is not present in the second, follow-up voice query;

in response to determining that the anaphora is present in the second, follow-up voice query, determining that the first voice query is associated with a plurality of possible entities;

generating a plurality of candidate queries, wherein each candidate query of the plurality of candidate queries is generated by replacing the pronoun in the second, follow-up voice query with a corresponding possible entity;

determining that a corresponding possible entity in a first candidate query of the plurality of candidate queries is an entity associated with the anaphora from the second, follow-up voice query, wherein the entity associated with the anaphora from the second, follow-up voice query is the entity that is not present in the second, follow-up voice query and is referred to by the pronoun in the second, follow-up voice query; and

based on determining that the corresponding possible entity in the first candidate query is the entity associated with the anaphora, providing the first candidate query to a search engine and obtaining search results for the first candidate query from the search engine.

18. The computer storage media of claim 17, further comprising:

identifying an answer for the first candidate query in the search results for the first candidate query; and

audibly presenting the answer in response to the second, follow-up voice query.

19. The computer storage media of claim 17, wherein determining that the first voice query is associated with a plurality of possible entities further comprises the step of:

analyzing results of the first voice query for a presence of an entity associated with the anaphora.

20. The computer storage media of claim 17, wherein determining that a corresponding possible entity in a first candidate query of the plurality of candidate queries is an entity associated with the anaphora from the second, follow-up voice query comprises:

providing the plurality of candidate queries to a ranking engine;

obtaining a ranking of the candidate queries from the ranking engine; and

determining that the first candidate query is a highest-ranked candidate query according to the ranking.