LINGUISTIC KEY NORMALIZATION

• US 20130151235A1
• Filed: 03/25/2009
• Published: 06/13/2013
• Est. Priority Date: 03/26/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Och, Franz Josef, Thayer, Ignacio E., Tsochandaridis, Ioannis, Genzel, Dmitriy
• Granted Patent: US 8,521,516 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/237 Lexical tools

Abstract

Systems, methods, and apparatuses including computer program products are provided for training machine learning systems. In some implementations, a method is provided. The method includes receiving a collection of phrases, normalizing a plurality of phrases of the collection of phrases, the normalizing being based at least in part on lexicographic normalizing rules, and generating a normalized phrase table including a plurality of key-value pairs, each key value pair includes a key corresponding to a normalized phrase and a value corresponding to one or more un-normalized phrases associated with the normalized key, each un-normalized phrase having one or more parameters.

Claims

1. A method comprising:

receiving a collection of phrases;

normalizing a plurality of phrases of the collection of phrases, the normalizing being based at least in part on lexicographic normalizing rules; and

generating a normalized phrase table including a plurality of key-value pairs, each key-value pair having a key that includes a normalized phrase and a value that includes one or more un-normalized phrases associated with the normalized phrase of the key and one or more parameters associated with each un-normalized phrase, the one or more parameters including a translation corresponding to the normalized phrase and a probability for the translation given the normalized phrase.

2. The method of claim 1, where normalizing each phrase of the plurality of phrases includes applying one or more normalizing rules, the one or more normalizing rules including rules normalizing based on a case and a morphology of the phrase.

5. The method of claim 1, where the one or more parameters associated with each un-normalized phrase include an identification of a language associated with the phrase.

6. The method of claim 1, where the normalized phrase table includes a plurality of normalized phrases corresponding to un-normalized phrases in a plurality of languages.

3-4. -4. (canceled)

7. A method comprising:

receiving a training phrase;

normalizing the training phrase according to one or more lexicographic normalization rules;

identifying the normalized training phrase in a normalized phrase table, the normalized phrase table including a plurality of key-value pairs, each key-value pair having a key that includes a normalized phrase and a value that includes one or more un-normalized phrases associated with the normalized phrase of the key and one or more parameters associated with each un-normalized phrase;

associating one or more weights to one or more un-normalized phrases associated with the key-value pair for the identified normalized training phrase in the normalized phrase table based on a relation of each associated un-normalized phrase to the received training phrase; and

training a machine learning model using the one or more un-normalized phrases and the associated one or more weights.

8. The method of claim 7, further comprising:

determining whether the received training phrase matches an un-normalized phrase associated with the identified normalized training phrase;

associating a first weight to the un-normalized phrase when the training phrase matches the un-normalized phrase associated with the identified normalized training phrase; and

associating a second weight to the un-normalized phrase when the training phrase does not match the un-normalized phrase associated with the identified normalized training phrase.

9. The method of claim 7, where associating one or more weights further comprises:

assigning a weight to each un-normalized phrase based on a distance measure between each un-normalized phrase and the received training phrase.

10. The method of claim 9, where the distance measure includes determining a distance vector having entries corresponding to the one or more lexicographic normalization rules.

11. The method of claim 7, where training the machine learning model includes:

using one or more un-normalized phrases and their associated assigned weights as particular feature functions for the received phrase in a machine learning model.

12. The method of claim 7, where training the machine learning model includes training a language model.

13. The method of claim 7, where training the machine learning model includes training a language identification model.

14. The method of claim 7, where training the machine learning model includes training a statistical machine translation model.

15. The method of claim 7, where identifying the normalized phrase includes identifying a particular chunk of a distributed normalized phrase table including a key value corresponding to the normalized training phrase and searching the identified chunk for the normalized training phrase.

16. A computer program product, encoded on a non-transitory computer-readable medium, operable to cause a data processing apparatus to perform operations comprising:

receiving a collection of phrases;

normalizing a plurality of phrases of the collection of phrases, the normalizing being based at least in part on lexicographic normalizing rules; and

generating a normalized phrase table including a plurality of key-value pairs, each key-value pair having a key that includes a normalized phrase and a value that includes one or more un-normalized phrases associated with the normalized phrase of the key and one or more parameters associated with each un-normalized phrase, the one or more parameters including a translation corresponding to the normalized phrase and a probability for the translation given the normalized phrase.

17. The computer program product of claim 16, where normalizing each phrase of the plurality of phrases includes applying one or more normalizing rules, the one or more normalizing rules including rules normalizing based on a case and a morphology of the phrase.

20. The computer program product of claim 16, where the one or more parameters associated with each un-normalized phrase include an identification of a language associated with the phrase.

21. The computer program product of claim 16, where the normalized phrase table includes a plurality of normalized phrases corresponding to un-normalized phrases in a plurality of languages.

18-19. -19. (canceled)

22. A computer program product, encoded on a non-transitory computer-readable medium, operable to cause a data processing apparatus to perform operations comprising:

receiving a training phrase;

normalizing the training phrase according to one or more lexicographic normalization rules;

identifying the normalized training phrase in a normalized phrase table, the normalized phrase table including a plurality of key-value pairs, each key-value pair having a key that includes a normalized phrase and a value that includes one or more un-normalized phrases associated with the normalized phrase of the key and one or more parameters associated with each un-normalized phrase;

associating one or more weights to one or more un-normalized phrases associated with the key-value pair for the identified normalized training phrase in the normalized phrase table based on a relation of each associated un-normalized phrase to the received training phrase; and

training a machine learning model using the one or more un-normalized phrases and the associated one or more weights.

23. The computer program product of claim 22, the computer program product further operable to cause the data processing apparatus to perform operations comprising:

determining whether the received training phrase matches an un-normalized phrase associated with the identified normalized training phrase;

associating a first weight to the un-normalized phrase when the training phrase matches the un-normalized phrase associated with the identified normalized training phrase; and

associating a second weight to the un-normalized phrase when the training phrase does not match the un-normalized phrase associated with the identified normalized training phrase.

24. The computer program product of claim 22, where associating one or more weights further comprises:

assigning a weight to each un-normalized phrase based on a distance measure between each un-normalized phrase and the received training phrase.

25. The computer program product of claim 24, where the distance measure includes determining a distance vector having entries corresponding to the one or more lexicographic normalization rules.

26. The computer program product of claim 22, where training the machine learning model includes:

using one or more un-normalized phrases and their associated assigned weights as particular feature functions for the received phrase in a machine learning model.

27. The computer program product of claim 22, where training the machine learning model includes training a language model.

28. The computer program product of claim 22, where training the machine learning model includes training a language identification model.

29. The computer program product of claim 22, where training the machine learning model includes training a statistical machine translation model.

30. The computer program product of claim 22, where identifying the normalized phrase includes identifying a particular chunk of a distributed normalized phrase table including a key value corresponding to the normalized training phrase and searching the identified chunk for the normalized training phrase.

31. A system comprising:

one or more computers configured to perform operations including;

receiving a collection of phrases;

normalizing a plurality of phrases of the collection of phrases, the normalizing being based at least in part on lexicographic normalizing rules; and

generating a normalized phrase table including a plurality of key-value pairs, each key-value pair having a key that includes a normalized phrase and a value that includes one or more un-normalized phrases associated with the normalized phrase of the key and one or more parameters associated with each un-normalized phrase, the one or more parameters including a translation corresponding to the normalized phrase and a probability for the translation given the normalized phrase.

32. A system comprising:

one or more computers configured to perform operations including;

receiving a training phrase;

normalizing the training phrase according to one or more lexicographic normalization rules;

identifying the normalized training phrase in a normalized phrase table, the normalized phrase table including a plurality of key-value pairs, each key-value pair having a key that includes a normalized phrase and a value that includes one or more un-normalized phrases associated with the normalized phrase of the key and one or more parameters associated with each un-normalized phrase;

associating one or more weights to one or more un-normalized phrases associated with the key-value pair for the identified normalized training phrase in the normalized phrase table based on a relation of each associated un-normalized phrase to the received training phrase; and

training a machine learning model using the one or more un-normalized phrases and the associated one or more weights.