Ranker selection for statistical natural language processing

• US 7,844,555 B2
• Filed: 11/13/2007
• Issued: 11/30/2010
• Est. Priority Date: 11/13/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Johnson, Mark, Toutanova, Kristina, Gao, Jianfeng, Andrew, Galen
• Primary Examiner(s): HOLMES, MICHAEL B
• Application Number:
  US11/938,811
• Publication Number:
  US 20090125501A1
• Time in Patent Office: 1,113 Days
• Field of Search: 706/15
• US Class Current: 706/15
• CPC Class Codes: G06F 40/216 using statistical methods

Abstract

Systems and methods for selecting a ranker for statistical natural language processing are provided. One disclosed system includes a computer program configured to be executed on a computing device, the computer program comprising a data store including reference performance data for a plurality of candidate rankers, the reference performance data being calculated based on a processing of test data by each of the plurality of candidate rankers. The system may further include a ranker selector configured to receive a statistical natural language processing task and a performance target, and determine a selected ranker from the plurality of candidate rankers based on the statistical natural language processing task, the performance target, and the reference performance data.

Claims

1. A system for selecting a ranker for statistical natural language processing, the system comprising a computer program configured to be executed on a computing device, the computer program including:

a data store including reference performance data for a plurality of candidate rankers, the reference performance data being calculated based on a processing of test data by each of the candidate rankers; and

a ranker selector configured to receive an SNLP task and one or more performance targets, and determine a selected ranker from the plurality of candidate rankers based on the SNLP task, the performance target, and the reference performance data.

2. The system of claim 1, wherein the computer program is an application program interface configured to be a communication intermediary between an application program and the ranker selector.

3. The system of claim 1, further comprising a graphical user interface including an SNLP task input tool configured to receive user input of the SNLP task.

4. The system of claim 3, wherein the graphical user interface further includes a performance targets input tool configured to receive user input of the performance targets.

5. The system of claim 3, wherein the graphical user interface further includes an output pane configured to display the selected ranker.

6. The system of claim 3, wherein the graphical user interface further includes a test data input tool configured to receive user input of the test data, a load candidate ranker tool configured to receive user input of one or more candidate rankers, a load reference performance data tool configured to receive user input of reference performance data.

7. The system of claim 3, wherein the graphical user interface further includes a recommend ranker selector configured to receive a user request to determine the selected ranker.

8. The system of claim 1, wherein the one or more performance targets are selected from the group consisting of accuracy, memory footprint, processing time, development time, model sparsity, and model training time.

9. The system of claim 1, wherein the candidate ranker is selected from the group consisting of maximum entropy estimation with L₂ regularization ranker, averaged perceptron ranker, boosting ranker, maximum entropy estimation with L₁ regularization ranker, and BLasso ranker.

10. The system of claim 1, wherein the SNLP task is formulated as a ranking problem under the framework of a linear model and is selected from the group consisting of parse selection, language model adaptation, word segmentation, part-of-speech tagging, text categorization, text clustering, lexical acquisition, word-sense disambiguation, word class induction, syntactic parsing, semantic interpretation, grammar induction, and machine translation.

11. The system of claim 1, further including a test engine configured to receive the test data and to generate the reference performance data by processing the test data with each of the candidate rankers.

12. A method for selecting a ranker for statistical natural language processing using a computing device, the method comprising:

providing a plurality of candidate rankers;

providing reference performance data, the reference performance data being calculated based on a processing of predefined test data by each of the plurality of candidate rankers;

receiving an SNLP task and one or more performance targets;

receiving a user request to select a selected ranker from among the plurality of candidate rankers; and

determining the selected ranker from the plurality of candidate rankers based on the SNLP task, the performance target, and the reference performance data.

13. The method of claim 12, further comprising:

displaying the selected ranker on a display associated with the computing device.

14. The method of claim 12, wherein the one or more performance targets are selected from the group consisting of accuracy, memory footprint, processing time, development time, model sparsity, and model training time.

15. The method of claim 12, wherein the candidate ranker is selected from the group consisting of maximum entropy estimation with L₂ regularization ranker, averaged perceptron ranker, boosting ranker, maximum entropy estimation with L₁ regularization ranker, and BLasso ranker.

16. The method of claim 12, wherein the SNLP task is selected from the group consisting of parse selection, language model adaptation, word segmentation, part-of-speech tagging, text categorization, text clustering, lexical acquisition, word-sense disambiguation, word class induction, syntactic parsing, semantic interpretation, grammar induction, and machine translation.

17. The method of claim 12, wherein receiving an SNLP task and one or more performance targets includes receiving an SNLP task via an SNLP task input tool of a GUI of a computer program, and receiving one or more performance targets via a performance targets input tool of the GUI.

18. The method of claim 12, wherein determining the selected ranker from the plurality of candidate rankers includes comparing a measured parameter from the reference performance data for each of the plurality of candidate rankers with a desired parameter from the performance targets.

19. A system for selecting a ranker for statistical natural language processing, the system comprising a computer program executable on a computing device to display a graphical user interface on a display associated with the computing device, the graphical user interface comprising:

an SNLP task input tool configured to receive user input indicating an SNLP task, a performance targets input tool configured to receive user input indicating one or more performance targets;

a recommend ranker selector configured to receive user request to determine a selected ranker, based on the SNLP task, performance targets, and stored reference performance data; and

an output pane configured to display the selected ranker, upon determination of the selected ranker by the computer program.

20. The system of claim 19, wherein the graphical user interface further comprises:

a load test data tool configured to receive user input of test data;

a load candidate ranker tool configured to receive user input of one or more candidate rankers; and

a load reference performance data tool configured to receive user input of reference performance data.