Search ranking based on natural language query patterns

• US 9,396,235 B1
• Filed: 12/13/2013
• Issued: 07/19/2016
• Est. Priority Date: 12/13/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wald, Jonathan, Mezheritsky, Ilya
• Primary Examiner(s): Nguyen, Cam-Linh
• Application Number:
  US14/105,460
• Time in Patent Office: 949 Days
• Field of Search: 707/723, 707/713, 707/769
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/90332 Natural language query formulation or dialogue systems

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining whether to provide a factual answer to a user'"'"'s query. In one aspect, a method includes obtaining a set of query patterns associated with a particular fact. A subset of natural language query patterns can be identified from the set of query patterns. A likelihood that a query corresponding to at least one of the query patterns from the set of query patterns is a fact-seeking query can be determined based at least in part on comparing a first query count associated with the subset of natural language query patterns with a second query count associated with the set of query patterns. The likelihood can be provided as a signal for ranking search results.

Claims

1. A computer-implemented method comprising:

obtaining, by a search engine, a set of query patterns that are associated with searching for a particular fact, the set of query patterns including a first subset of one or more query patterns that are expressed in a natural language format, and a second subset of one or more query patterns that are not expressed in a natural language format;

determining, by the search engine, (i) a first quantity of query patterns that occur in the set of query patterns that are associated with searching for the particular fact, and (ii) a second quantity of query patterns that occur in the first subset of query patterns that are expressed in a natural language format;

determining, by the search engine, a likelihood that a particular query is a fact-seeking query based at least on comparing the first quantity and the second quantity, wherein a fact-seeking query is a query for which a factual answer is presented adjacent to one or more search results, in an answer area of a search results page that is generated in response to the query; and

using, by the search engine, the likelihood as a signal for ranking search results.

2. The method of claim 1, wherein the first quantity and the second quantity correspond to counts of voice queries.

3. The method of claim 1, wherein the first quantity and the second quantity correspond to counts of textual queries.

4. The method of claim 1, further comprising identifying the first subset of query patterns that are expressed in a natural language format, including:

for each query pattern in the set of query patterns, determining a likelihood that the respective query pattern corresponds to a natural language query pattern; and

selecting the respective query pattern when the likelihood that the respective query pattern corresponds to a natural language query pattern satisfies a threshold.

5. The method of claim 1, wherein obtaining a set of query patterns that are associated with searching for a particular fact comprises:

obtaining a plurality of fact-seeking queries;

determining that a set of the fact-seeking queries relate to the particular fact; and

generating the set of query patterns from the set of the fact-seeking queries that relate to the particular fact.

6. The method of claim 1, wherein the likelihood corresponds to a ratio of the first quantity over the second quantity.

7. The method of claim 1, wherein the first quantity is based on a number of queries stored in a query log that are related to the particular fact and that are classified as having a natural language format.

8. The method of claim 1, wherein each query pattern that is expressed in a natural language format comprises a query pattern that is classified as having a form in which a user would phrase a question when speaking to another person.

9. A computer-implemented method comprising:

receiving, at a search engine and from a client device, a query;

determining, by the search engine, a particular fact associated with the received query;

based on the particular fact, obtaining a likelihood that the received query is a fact-seeking query, wherein the likelihood is based at least on comparing (i) a first quantity of query patterns that occur in a set of query patterns that are associated with searching for the particular fact with (ii) a second quantity of query patterns that occur in a first subset of the set of query patterns, the first subset of the set of query patterns including one or more query patterns that are expressed in a natural language format;

based on determining that the likelihood satisfies a threshold, adding, by the search engine, a factual answer to a set of search results for the query; and

providing, by the search engine and to the client device, the set of search results in response to the query.

10. The method of claim 9, wherein the factual answer specifies the particular fact.

11. The method of claim 9, further comprising ranking the factual answer with the search results using the likelihood and ranking scores for each of the search results, wherein the search results and the factual answer are ordered according to the ranking.

12. The method of claim 9, wherein:

the received query comprises a voice query;

the first quantity and the second quantity correspond to counts of voice queries; and

the threshold corresponds to voice-based queries.

13. The method of claim 9, wherein:

the received query comprises a textual query;

the first quantity and the second quantity correspond to counts of textual queries; and

the threshold corresponds to textual queries.

14. The method of claim 9, wherein each query pattern that is expressed in a natural language format comprises a query pattern that is classified as having a form in which a user would phrase a question when speaking to another person.

15. A system, comprising:

a data processing apparatus; and

a memory storage apparatus in data communication with the data processing apparatus, the memory storage apparatus storing instructions executable by the data processing apparatus and that upon such execution cause the data processing apparatus to perform operations comprising;

obtaining, by a search engine, a set of query patterns that are associated with searching for a particular fact, the set of query patterns including a first subset of one or more query patterns that are expressed in a natural language format, and a second subset of one or more query patterns that are not expressed in a natural language format;

determining, by the search engine, (i) a first quantity of query patterns that occur in the set of query patterns that are associated with searching for the particular fact, and (ii) a second quantity of query patterns that occur in the first subset of query patterns that are expressed in a natural language format;

determining, by the search engine, a likelihood that a particular query is a fact-seeking query based at least on comparing the first quantity and the second quantity, wherein a fact-seeking query is a query for which a factual answer is presented adjacent to one or more search results, in an answer area of a search results page that is generated in response to the query; and

using, by the search engine, the likelihood as a signal for ranking search results.

16. The system of claim 15, wherein the operations further comprise identifying the first subset of query patterns that are expressed in a natural language format, including:

for each query pattern in the set of query patterns, determining a likelihood that the respective query pattern corresponds to a natural language query pattern; and

selecting the respective query pattern when the likelihood that the respective query pattern corresponds to a natural language query pattern satisfies a threshold.

17. The system of claim 15, wherein obtaining a set of query patterns that are associated with searching for a particular fact comprises:

obtaining a plurality of fact-seeking queries;

determining that a set of the fact-seeking queries relate to the particular fact; and

generating the set of query patterns from the set of the fact-seeking queries that relate to the particular fact.

18. The system of claim 15, wherein the likelihood corresponds to a ratio of the first quantity over the second quantity.