Structured data translation apparatus, system and method

• US 8,306,807 B2
• Filed: 08/17/2009
• Issued: 11/06/2012
• Est. Priority Date: 08/17/2009
• Status: Active Grant

• Current Assignee: Ntrepid, LLC
• Original Assignee: N T Repid Corporation
• Inventors: Van Guilder, Linda C., Doyon, Jennifer B., Martinka, Michael E., Phillips, Jon B., Concepcion, Kristian J., Muir, Robert C.
• Primary Examiner(s): Vo, Huyen X.
• Application Number:
  US12/461,574
• Publication Number:
  US 20110040552A1
• Time in Patent Office: 1,177 Days
• Field of Search: 704 1- 10, 704/177, 704/251, 704/255, 704/257
• US Class Current: 704/4
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

An apparatus is provided that includes a processor and a memory. The processor is configured to determine a category for a group of isolated noun phrases in a structured or semi-structured data source stored in the memory. The group of isolated noun phrases includes one or more isolated noun phrases. The processor is also configured to translate the group of isolated noun phrases from a source language to a target language using a category-driven isolated noun phrase translation. The determination of the category and the category-driven isolated noun phrase translation are performed based on context derived from the group of isolated noun phrases.

Claims

1. An apparatus, comprising:

a processor; and

a memory, wherein the processor is configured to

determine a category for a group of isolated noun phrases in a structured or semi-structured data source stored in the memory, wherein the group of isolated noun phrases comprises one or more isolated noun phrases; and

translate the group of isolated noun phrases from a source language to a target language using a category-driven isolated noun phrase translation,

wherein the determination of the category and the category-driven isolated noun phrase translation are performed based on context derived from the group of isolated noun phrases;

wherein the processor is further configured to determine the category of the group of isolated noun phrases based on an automatic column categorization that identifies the most likely category of the group of isolated noun phrases that is obtained by combining results obtained for individual cells of the group of isolated noun phrases into a composite score for each category.

2. The apparatus of claim 1, wherein the processor is further configured to determine the most likely category for the group of isolated noun phrases as the category with the highest composite score.

3. The apparatus of claim 1, wherein the processor is further configured to only select the category with the highest composite score when the category with the highest composite score exceeds the category with the second highest composite score by a predetermined threshold.

4. The apparatus of claim 1, wherein the processor is further configured to determine the category of each individual cell using feature extraction and Bayesian statistics, or a Bayesian statistical model trained on data for which category labels are known.

5. The apparatus of claim 4, wherein features used for classification comprise one or more of data type characteristics, morpho-syntactic information, semantic features and categories, local distributional statistics and pattern-based features.

6. The apparatus of claim 1, wherein the processor is further configured to perform a hybrid category-driven isolated noun phrase transfer when a result for translating an isolated noun phrase from the source language to the target language is not found using the category-driven isolated noun phrase translation based on finding the isolated noun phrase in one or more translation memories or one or more dictionaries.

7. The apparatus of claim 6, wherein the processor is further configured to employ one or more of a transliteration only strategy, a key word replacement plus transliteration strategy, a category-driven lexical transfer strategy, a category-driven shallow syntactic transfer strategy or a statistical model translation strategy.

8. The apparatus of claim 1, wherein the processor is further configured to:

employ a shallow syntactic transfer strategy comprising inputting category-specific rules;

produce a translation graph by applying the category-specific rules; and

choose the best path through the translation graph.

9. The apparatus of claim 1, wherein the processor is further configured to:

determine whether transliteration is to be applied to all or part of an isolated noun phrase; and

when transliteration is to be applied, use a transliteration standard to replace indicated parts of the isolated noun phrase with appropriate transliterations.

10. The apparatus of claim 1, wherein the processor is further configured to divide cell data for the group of isolated noun phrases into tokens representing word-like units that are looked up in at least one of a dictionary and a translation memory and associated with a set of grammatical and semantic features.

11. The apparatus of claim 10, wherein the processor is further configured to:

inspect each token to determine whether the respective token should be submitted to morphological analysis;

when it is determined that a token needs morphological analysis, perform lemmatization to find a citation form of the token along with a dictionary and translation memory lookup to retrieve information associated with the citation form of the token; and

analyze affixes to determine grammatical characteristics marked by inflectional affixes.

12. A computer-implemented method, comprising:

determining, by a processor, a category for a group of isolated noun phrases in a data source, wherein the group of isolated noun phrases comprises one or more isolated noun phrases; and

translating, by the processor, the group of isolated noun phrases from a source language to a target language using a category-driven isolated noun phrase translation,

wherein the determination of the category and the category-driven isolated noun phrase translation are performed based on context derived from the group of isolated noun phrases,

wherein the category is determined based on an automatic column categorization that identifies the most likely category of the group of isolated noun phrases that is obtained by combining results obtained for individual cells of the group of isolated noun phrases into a composite score for each category.

13. The apparatus of claim 12, wherein the most likely category for the group of isolated noun phrases is determined as the category with the highest composite score.

14. The computer-implemented method of claim 12, wherein the category with the highest composite score is only selected when the category with the highest composite score exceeds the category with the second highest composite score by a predetermined threshold.

15. The computer-implemented method of claim 12, wherein the category of each individual cell is determined using feature extraction and Bayesian statistics, or a Bayesian statistical model trained on data for which category labels are known.

16. The computer-implemented method of claim 15, wherein features used for classification comprise one or more of data type characteristics, morpho-syntactic information, semantic features and categories, local distributional statistics and pattern-based features.

17. The computer-implemented method of claim 12, further comprising:

performing a hybrid category-driven isolated noun phrase transfer when a result for translating an isolated noun phrase from the source language to the target language is not found using the category-driven isolated noun phrase translation based on finding the isolated noun phrase in one or more translation memories or one or more dictionaries.

18. The computer-implemented method of claim 17, wherein the translating further comprises employing one or more of a transliteration only strategy, a key word replacement plus transliteration strategy, a category-driven lexical transfer strategy, a category-driven shallow syntactic transfer strategy and a statistical model translation strategy.

19. The computer-implemented method of claim 12, wherein the translating further comprises employing a shallow syntactic transfer strategy comprising

inputting category-specific rules,

producing a translation graph by applying the category-specific rules, and

choosing the best path through the translation graph.

20. The computer implemented method of claim 12, further comprising:

determining whether transliteration is to be applied to all or part of an isolated noun phrase; and

when transliteration is to be applied, using a transliteration standard to replace indicated parts of the isolated noun phrase with appropriate transliterations.

21. The computer-implemented method of claim 12, further comprising:

dividing cell data for the group of isolated noun phrases into tokens representing word-like units that are looked up in at least one of a dictionary and a translation memory; and

associating the tokens with a set of grammatical and semantic features.

22. The computer-implemented method of claim 21, further comprising:

inspecting each token to determine whether the respective token should be submitted to morphological analysis;

when it is determined that a token needs morphological analysis, performing lemmatization to find a citation form of the token along with a dictionary and translation memory lookup to retrieve information associated with the citation form of the token; and

analyzing affixes to determine grammatical characteristics marked by inflectional affixes.

23. A computer program embodied on a non-transitory computer-readable medium, the program configured to control a processor to perform a process, the process comprising:

determining a category for a group of isolated noun phrases in a data source, wherein the group of isolated noun phrases comprises one or more isolated noun phrases; and

translating the group of isolated noun phrases from a source language to a target language using a category-driven isolated noun phrase translation,

wherein the determination of the category and the category-driven isolated noun phrase translation are performed based on context derived from the group of isolated noun phrases,

wherein the category is determined based on an automatic column categorization that identifies the most likely category of the group of isolated noun phrases that is obtained by combining results obtained for individual cells of the group of isolated noun phrases into a composite score for each category.

24. An apparatus, comprising:

a processor; and

memory, the processor comprising

determining means for determining a category for a group of isolated noun phrases in a data source, wherein the group of isolated noun phrases comprises one or more isolated noun phrases; and

translating means for translating the group of isolated noun phrases from a source language to a target language using a category-driven isolated noun phrase translation,

wherein the determination of the category and the category-driven isolated noun phrase translation are performed based on context derived from the group of isolated noun phrases; and

determining means for determining the category of the group of isolated noun phrases based on an automatic column categorization that identifies the most likely category of the group of isolated noun phrases that is obtained by combining results obtained for individual cells of the group of isolated noun phrases into a composite score for each category.