Retrieval of domain relevant phrase tables

• US 9,582,499 B2
• Filed: 04/14/2014
• Issued: 02/28/2017
• Est. Priority Date: 04/14/2014
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Mathur, Prashant, Venkatapathy, Sriram, Cancedda, Nicola
• Primary Examiner(s): ALBERTALLI, BRIAN LOUIS
• Application Number:
  US14/252,194
• Publication Number:
  US 20150293910A1
• Time in Patent Office: 1,051 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 40/44 Statistical methods, e.g. probability models

Abstract

A method for generating a phrase table for a target domain includes receiving a source corpus for a target domain and, for each of a set of comparative domain phrase tables, computing a measure of similarity between the source corpus and the comparative domain phrase table. Based on the computed similarity measures, a subset of the comparative domain phrase tables may be identified from the set of comparative domain phrase tables, and/or weights for combining them, and a phrase table is generated for the target domain based on the at least a subset of phrase tables.

Claims

1. A method for generating a phrase table for a target domain comprising:

receiving a source corpus for a target domain;

for each of a set of comparative domain phrase tables, computing a measure of similarity between the source corpus and the comparative domain phrase table, the measure of similarity being computed as a function of counts of n-grams of each of a plurality of sizes in the source corpus that are also present in the respective phrase table;

based on the computed similarity measures, identifying a subset of the comparative domain phrase tables from the set of comparative domain phrase tables; and

generating a phrase table for the target domain based on the subset of phrase tables;

wherein the computing of the similarity measures, identifying the subset of the phrase tables, and generating the phrase table is performed with a computer processor.

2. The method of claim 1, wherein each similarity measure may be computed as a function of

3. The method of claim 2, wherein p is 1.

4. The method of claim 2, wherein q is at least 3.

5. The method of claim 2, wherein r is at least 3.

6. The method of claim 2, wherein the similarity (SIM-PT (PT,S) is computed as:

7. The method of claim 2, wherein the source corpus includes at least 50 n-grams of each size n.

8. The method of claim 1 wherein the generating of the phrase table for the target domain comprises combining the subset of phrase tables in a weighted combination in which each of the comparative domain phrase tables in the subset is weighted as a function of the measure of similarity between the source corpus for the target domain and the respective comparative domain phrase table.

9. The method of claim 8, further comprising normalizing the similarity measures of the comparative domain phrase tables in the subset of phrase tables, and wherein in the combining of the subset of phrase tables, each of the comparative domain phrase tables in the subset is weighted by the respective normalized similarity measure.

10. The method of claim 1, wherein each of the comparative domain phrase tables includes biphrase features for each of a set of biphrases, each biphrase including a source phrase and a corresponding target phrase, the biphrase features having been derived from a parallel corpus of source and target text strings for the respective comparative domain.

11. The method of claim 10, wherein the generating of the phrase table for the target domain comprises for each of the set of biphrase features of one of the biphrases in the set of biphrases, computing a weighted combination of feature values of each of the subset of phrase tables.

12. The method of claim 10, wherein the biphrase features include features selected from the group consisting of lexical features and phrasal features.

13. The method of claim 1, wherein the method further comprises encoding the set of comparative domain phrase tables as a finite state transducer, the computing a measure of similarity between the source corpus and the comparative domain phrase table comprising inputting a phrase extracted from the source corpus to the finite state transducer and outputting, from the finite state transducer, indices of each of the comparative domain phrases tables which include the phrase.

15. The method of claim 1, wherein the set of comparative domain phrase tables comprises at least five comparative domain phrase tables The method of claim 1, wherein the method is performed without access to a parallel corpus in the target domain.

16. The method of claim 1, wherein the subset of comparative domain phrase tables comprises at least three comparative domain phrase tables.

17. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1.

18. A system for generating a phrase table for a target domain comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

14. A method, for generating a phrase table for a target domain comprising:

receiving a source corpus for a target domain;

for each of a set of comparative domain phrase tables, computing a measure of similarity between the source corpus and the comparative domain phrase table, wherein the computing of the measure of similarity between the source corpus and the comparative domain phrase table comprises computing an occurrence of each of a set of phrases in the source corpus in biphrases of each of the phrase tables;

based on the computed similarity measures, identifying a subset of the comparative domain phrase tables from the set of comparative domain phrase tables; and

generating a phrase table for the target domain based on the subset of phrase tables;

wherein the computing of the similarity measures, identifying the subset of the phrase tables, and generating the phrase table is performed with a computer processor.

19. A system for generating a phrase table for a target domain comprising:

a similarity computation component which, for each of a set of comparative domain phrase tables, computes a measure of similarity between an input source corpus and the comparative domain phrase table;

a multi-model computation component which identifies a subset of the comparative domain phrase tables from the set of comparative domain phrase tables based on the computed similarity measures and generates a phrase table for the target domain based on the subset of phrase tables; and

a processor which implements the similarity computation component and the multi-model computation component.

20. A method for generating a phrase table for a target domain comprising:

receiving a source corpus for a target domain;

for each of a set of comparative domain phrase tables, computing a measure of similarity between the source corpus and the comparative domain phrase table;

combining at least a subset of the comparative domain phrase tables from the set of comparative domain phrase tables in a weighted combination, weights for the combination being based on the computed similarity measures; and

wherein at least one of the computing of the similarity measures, identifying the subset of the phrase tables, and combining is performed with a computer processor.