Query language identification

• US 9,727,605 B1
• Filed: 04/09/2013
• Issued: 08/08/2017
• Est. Priority Date: 04/19/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lopiano, Fabio
• Primary Examiner(s): Andersen, Kristopher
• Application Number:
  US13/859,589
• Time in Patent Office: 1,582 Days
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/3338 Query expansion
• CPC Class Codes: G06F 40/263 Language identification

Abstract

Methods, systems, and apparatus, including computer program products, for identifying the language of a search query. In one embodiment, the language of each term of a query is determined from the query terms and the language of the user interface a user used to enter the query. In another embodiment, an automatic interface language classifier is generated from a collection of past queries each submitted by a user. In some embodiments, a score is determined for each of multiple languages, each score indicating a likelihood that the query language is the corresponding one of the multiple languages.

Claims

1. A method comprising:

providing, by a system comprising one or more computers, a plurality of user interfaces through which search queries are received, wherein each user interface is in a respective interface language, and wherein each interface language is a natural language in which a respective user interface presents information;

maintaining a collection of query records, wherein the collection of query records includes distinct subsets of query records, wherein each distinct subset of query records is associated with a respective user interface of the plurality of user interfaces, wherein each query record associates a past query with one or more result documents, and wherein each result document has an associated natural language;

classifying each past query in the collection of query records based at least on;

(i) the interface language of the user interface through which the past query was received, and (ii) at least one of;

(a) a natural language of the one or more result documents associated with the past query, or (b) the natural language of one or more result documents that were selected;

generating an initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces based on the classifying, wherein the initial distribution indicates, for each user interface of the plurality of user interfaces and for each of multiple natural languages, what proportion of the past queries from the plurality of query records were in the language for the interface;

generating, based at least on the initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces, an interface language classifier that is trained to predict, for a given user interface and a given language, a proportion of queries that are received through the given user interface that are likely in the given language;

receiving, in the system, through a first user interface of the plurality of user interfaces, a search query comprising one or more query terms;

using the interface language classifier, that was generated based at least on the initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces, to determine a likelihood that the search query is in a particular natural language of the multiple natural languages, given that the first user interface is the user interface that received the query; and

providing one or more results responsive to the search query received through the first user interface, wherein the one or more results comprises results in a most likely natural language of the search query, which is automatically determined by the interface language classifier from the multiple natural languages.

2. The method of claim 1, further comprising tuning the interface language classifier based on queries that were correctly or incorrectly classified by the classifying until the tuned interface language classifier is sufficiently accurate.

3. The method of claim 2, further comprising:

using the tuned interface language classifier to reclassify each of a plurality of past queries in the collection of records;

comparing each of the plurality of past queries that are reclassified to respective labeled queries; and

determining an accuracy of the tuned interface language classifier based on a result of the comparing.

4. The method of claim 1, further comprising:

receiving input, from a user, identifying a language of a particular past query that includes one or more words based on an examination, by the user, of the one or more words; and

generating, from the particular past query, a labeled query according to the input.

5. The method of claim 1, further comprising:

using the interface classifier to generate a respective interface classifier vector for each of the plurality of user interfaces.

6. The method of claim 1, further comprising:

determining, by the system, a query language of the search query based at least in part on;

the search query, the interface language of the first user interface, and the distinct subset of query records that is associated with the first user interface, the query language being a natural language.

7. A computer program product, encoded on one or more non-transitory computer storage media, comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

providing a plurality of user interfaces through which search queries are received, wherein each user interface is in a respective interface language, and wherein each interface language is a natural language in which a respective user interface presents information;

maintaining a collection of query records, wherein the collection of query records includes distinct subsets of query records, wherein each distinct subset of query records is associated with a respective user interface of the plurality of user interfaces, wherein each query record associates a past query with one or more result documents, and wherein each result document has an associated natural language;

classifying each past query in the collection of query records based at least on;

(i) the interface language of the user interface through which the past query was received, and (ii) at least one of;

(a) a natural language of the one or more result documents associated with the past query, or (b) the natural language of one or more result documents that were selected;

generating an initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces based on the classifying, wherein the initial distribution indicates, for each user interface of the plurality of user interfaces and for each of multiple natural languages, what proportion of the past queries from the plurality of query records were in the language for the interface;

generating, based at least on the initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces, an interface language classifier that is trained to predict, for a given user interface and a given language, a proportion of queries that are received through the given user interface that are likely in the given language;

receiving, in the system, through a first user interface of the plurality of user interfaces, a search query comprising one or more query terms;

using the interface language classifier, that was generated based at least on the initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces, to determine a likelihood that the search query is in a particular natural language of the multiple natural languages, given that the first user interface is the user interface that received the query; and

providing one or more results responsive to the search query received through the first user interface, wherein the one or more results comprises results in a most likely natural language of the search query, which is automatically determined by the interface language classifier from the multiple natural languages.

8. The product of claim 7, the operations further comprising:

tuning the interface language classifier based on queries that were correctly or incorrectly classified by the classifying until the tuned interface language classifier is sufficiently accurate.

9. The product of claim 8, the operations further comprising:

using the tuned interface language classifier to reclassify each of a plurality of past queries in the collection of records;

comparing each of the plurality of past queries that are reclassified to respective labeled queries; and

determining an accuracy of the tuned interface language classifier based on a result of the comparing.

10. The product of claim 7, the operations further comprising:

receiving input, from a user, identifying a language of a particular past query that includes one or more words based on an examination, by the user, of the one or more words; and

generating, from the particular past query, a labeled query according to the input.

11. The product of claim 7, the operations further comprising:

using the interface classifier to generate a respective interface classifier vector for each of the plurality of user interfaces.

12. The product of claim 7, the operations further comprising:

determining, by the system, a query language of the search query based at least in part on;

the search query, the interface language of the first user interface, and the distinct subset of query records that is associated with the first user interface, the query language being a natural language.

13. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

providing a plurality of user interfaces through which search queries are received, wherein each user interface is in a respective interface language, and wherein each interface language is a natural language in which a respective user interface presents information;

maintaining a collection of query records, wherein the collection of query records includes distinct subsets of query records, wherein each distinct subset of query records is associated with a respective user interface of the plurality of user interfaces, wherein each query record associates a past query with one or more result documents, and wherein each result document has an associated natural language;

classifying each past query in the collection of query records based at least on;

(i) the interface language of the user interface through which the past query was received, and (ii) at least one of;

(a) a natural language of the one or more result documents associated with the past query, or (b) the natural language of one or more result documents that were selected;

generating an initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces based on the classifying, wherein the initial distribution indicates, for each user interface of the plurality of user interfaces and for each of multiple natural languages, what proportion of the past queries from the plurality of query records were in the language for the interface;

generating, based at least on the initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces, an interface language classifier that is trained to predict, for a given user interface and a given language, a proportion of queries that are received through the given user interface that are likely in the given language;

receiving, in the system, through a first user interface of the plurality of user interfaces, a search query comprising one or more query terms;

using the interface language classifier, that was generated based at least on the initial distribution of languages associated with the past queries for each user interface of the plurality of user interfaces, to determine a likelihood that the search query is in a particular natural language of the multiple natural languages, given that the first user interface is the user interface that received the query; and

providing one or more results responsive to the search query received through the first user interface, wherein the one or more results comprises results in a most likely natural language of the search query, which is automatically determined by the interface language classifier from the multiple natural languages.

14. The system of claim 13, the operations further comprising:

tuning the interface language classifier based on queries that were correctly or incorrectly classified by the classifying until the tuned interface language classifier is sufficiently accurate.

15. The system of claim 14, the operations further comprising:

using the tuned interface language classifier to reclassify each of a plurality of past queries in the collection of records;

comparing each of the plurality of past queries that are reclassified to respective labeled queries; and

determining an accuracy of the tuned interface language classifier based on a result of the comparing.

16. The system of claim 13, the operations further comprising:

receiving input, from a user, identifying a language of a particular past query that includes one or more words based on an examination, by the user, of the one or more words; and

generating, from the particular past query, a labeled query according to the input.

17. The system of claim 13, the operations further comprising:

using the interface classifier to generate a respective interface classifier vector for each of the plurality of user interfaces.

18. The system of claim 13, the operations further comprising:

determining, by the system, a query language of the search query based at least in part on;

the search query, the interface language of the first user interface, and the distinct subset of query records that is associated with the first user interface, the query language being a natural language.