NAMED ENTITY TRANSLITERATION USING CORPORATE CORPRA

• US 20100106484A1
• Filed: 10/21/2008
• Published: 04/29/2010
• Est. Priority Date: 10/21/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: U, Raghavendra Udupa, Khrisnan, Saravanan, Kumaran, Arumugam
• Granted Patent: US 8,560,298 B2
• Time in Patent Office: Days
• US Class Current: 704/5
• CPC Class Codes: G06F 40/129 Handling non-Latin characters, e.g. kana-to-kanji conversion
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/45 Example-based machine translation; Alignment

Abstract

A document in a first language and an additional document in a second language may be reviewed. It may be determined if the additional document is sufficiently similar to the document. If the additional document is determined sufficiently similar to the document, a named entity in the document may be selected. The method may search for a similar named entity by comparing the named entity to a word in the additional document and determining if the named entity and word are sufficiently similar. If a similar word to the named entity is located, the named entity and the similar named entities may be stored as name entity transliterations.

Claims

1. A method of mining multilingual named entity transliteration comprising:

Reviewing a document in a first language;

Reviewing an additional document in a second language;

Determining if the additional document is sufficiently similar to the document;

If the additional document is sufficiently similar to the document;

Selecting a named entity in the document;

Searching for a sufficiently similar named entity comprising comparing the named entity to a word in the additional document;

If a sufficiently similar word to the named entity is located, storing the named entity and the similar word as named entity transliterations.

2. The method of claim 1, wherein determining if the additional document is sufficiently similar to the document further comprises calculating a cross language similarity score.

3. The method of claim 2, wherein the cross language similarity score is calculated using a Kullback-Leibler divergence.

4. The method of claim 2, wherein the cross language similarity score is calculated for a plurality of document and additional document pairs.

5. The method of claim 4, further comprising selecting the document pair with the highest similarity score.

6. The method of claim 1, wherein searching for a similar named entity comprises calculating a cross language similarity score for the word and the named entity.

7. The method of claim 1, wherein the cross language similarity score measures the degree of transliteration equivalence between the named entity and the word.

8. The method of claim 7, wherein the cross language similarity score is calculated for a plurality of named entity pairs wherein named entity pairs comprise the named entity and the word in the additional document.

9. The method of claim 8, wherein the word in the additional document is sequentially selected from a group of words in the additional document wherein the group does not include prepositions, verbs or adjectives in the additional document.

10. The method of claim 9, wherein the named entity pair that has the cross language score at a maximum is selected as transliterations of each other.

11. A computer storage medium comprising computer executable instructions for mining multilingual named entity transliteration, the computer executable instructions comprising instructions for:

Reviewing a document in a first language;

Reviewing an additional document in a second language;

Determining if the additional document is sufficiently similar to the document by calculating a cross language similarity score;

If the additional document is sufficiently similar to the document;

Selecting a named entity in the document;

Searching for a sufficiently similar named entity comprising comparing the named entity to a word in the additional document;

If a sufficiently similar word to the named entity is located, storing the named entity and the similar word as named entity transliterations.

12. The computer storage medium of claim 11, wherein the cross language similarity score is calculated using a Kullback-Leibler divergence.

13. The computer storage medium of claim 12, wherein the cross language similarity score is calculated for a plurality of document and additional document pairs and selecting the document pair with the highest similarity score.

14. The computer storage medium of claim 11, wherein searching for a similar named entity comprises calculating a cross language similarity score for the word and the named entity wherein the cross language similarity score measures the degree of transliteration equivalence between the named entity and the word.

15. The computer storage medium of claim 14, wherein the cross language similarity score is calculated for a plurality of named entity pairs wherein named entity pairs comprise the named entity and the word in the additional document.

16. The computer storage medium of claim 15, wherein the word in the additional document is sequentially selected from a group of words in the additional document wherein the group does not include prepositions, verbs or adjectives in the additional document.

17. The computer storage medium of claim 16, wherein the named entity pair that has the cross language score at a maximum is selected as transliterations of each other.

18. A computer system comprising a processor for executing computer executable instructions for mining multilingual named entity transliteration, a memory in communication with the processor and an input output circuit, the computer executable instructions comprising instructions for:

Reviewing a document in a first language;

Reviewing an additional document in a second language;

Determining if the additional document is sufficiently similar to the document by calculating a cross language similarity score wherein the cross language similarity score is calculated using a Kullback-Leibler divergence;

If the additional document is sufficiently similar to the document;

Selecting a named entity in the document;

Searching for a sufficiently similar named entity comprising comparing the named entity to a word in the additional document;

If a sufficiently similar word to the named entity is located, storing the named entity and the similar word as named entity transliterations.

19. The computer system of claim 18, wherein the cross language similarity score is calculated for a plurality of document and additional document pairs and selecting the document pair with the highest similarity score.

20. The computer system of claim 18, wherein searching for a similar named entity comprises calculating a cross language similarity score for the word and the named entity wherein:

the cross language similarity score measures the degree of transliteration equivalence between the named entity and the word

the word in the additional document is sequentially selected from a group of words in the additional document wherein the group does not include prepositions, verbs or adjectives in the additional document;

wherein the cross language similarity score is calculated for a plurality of named entity pairs wherein named entity pairs comprise the named entity and the word in the additional document