Information Extraction in a Natural Language Understanding System

• US 20080310718A1
• Filed: 06/18/2007
• Published: 12/18/2008
• Est. Priority Date: 06/18/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Balchandran, Rajesh, Boyer, Linda M., Purdy, Gregory
• Granted Patent: US 8,521,511 B2
• Time in Patent Office: Days
• US Class Current: 382/176
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30

Abstract

A method of extracting information from text within a natural language understanding system can include processing a text input through at least one statistical model for each of a plurality of features to be extracted from the text input. For each feature, at least one value can be determined, at least in part, using the statistical model associated with the feature. One value for each feature can be combined to create a complex information target. The complex information target can be output.

Claims

1. A method of extracting information from text within a natural language understanding system, the method comprising:

processing a text input through at least one statistical model for each of a plurality of features to be extracted from the text input, wherein, for each feature, at least one value is determined, at least in part, using the statistical model associated with the feature;

combining one value for each feature to create a complex information target; and

outputting the complex information target.

2. The method of claim 1, wherein at least one statistical model outputs a plurality of candidate values for a feature that are ranked according to confidence score, wherein combining further comprises:

selecting a value that has a highest confidence score from the plurality of candidate values for each feature having a plurality of candidate values; and

combining the selected values to form the complex information target.

3. The method of claim 1, further comprising:

searching the text input for at least one named entity; and

determining a value for a feature according to the at least one named entity located within the text input.

4. The method of claim 1, wherein a selected one of the plurality of features is associated with a plurality of optimized statistical models, the method further comprising:

determining a value for a first feature in the text input; and

selecting one of the plurality of optimized statistical models associated with the selected feature according to the value determined for the first feature.

5. The method of claim 4, further comprising selecting the selected optimized statistical model to be a statistical model built using a subset of the plurality of features that correspond to the determined value for the first feature.

6. The method of claim 1, further comprising:

comparing the complex information target with a plurality of allowable complex information targets; and

determining whether the complex information target is allowable according to the comparison.

7. The method of claim 6, further comprising selecting an alternate value for at least one feature of the complex information target from a plurality of candidate values determined for the at least one feature according to confidence score.

8. The method of claim 7, further comprising selecting the alternate value from the plurality of candidate values that conforms to an allowable complex information target.

9. The method of claim 1, further comprising:

comparing the complex information target with a list comprising at least one predicted complex information target, wherein each predicted complex information target is associated with an expected complex information target; and

when the complex information target matches a predicted complex information target, mapping the complex information target to the expected complex information target associated with the matched predicted complex information target.

10. A method of extracting information from text within a natural language understanding system, the method comprising:

determining a value for a first feature using a selected text processing technique;

selecting a statistical model from a plurality of statistical models associated with a second feature according to the value determined for the first feature;

determining a value for the second feature using the selected statistical model;

combining the values for the first feature and the second feature to form a complex information target; and

outputting the complex information target.

11. The method of claim 10, further comprising selecting the text processing technique to be a statistical model associated with the first feature.

12. The method of claim 10, further comprising selecting the text processing technique to be a named entity feature extraction technique.

13. The method of claim 10, further comprising:

comparing the complex information target with a plurality of allowable complex information targets; and

determining whether the complex information target is valid according to the comparison.

14. A computer program product comprising:

a computer-usable medium comprising computer-usable program code that extracts information from text within a natural language understanding system, the computer-usable medium comprising;

computer-usable program code that processes a text input through at least one statistical model for each of a plurality of features to be extracted from the text input, wherein, for each feature, at least one value is determined, at least in part, using the statistical model associated with the feature;

computer-usable program code that combines one value for each feature to create a complex information target; and

computer-usable program code that outputs the complex information target.

15. The computer program product of claim 14, wherein at least one statistical model outputs a plurality of candidate values for a feature that are ranked according to confidence score, wherein the computer-usable program code that combines further comprises:

computer-usable program code that selects a value that has a highest confidence score from the plurality of candidate values for each feature having a plurality of candidate values; and

computer-usable program code that combines the selected values to form the complex information target.

16. The computer program product of claim 14, wherein the computer-usable medium further comprises:

computer-usable program code that searches the text input for at least one named entity; and

computer-usable program code that determines a value for a feature according to the at least one named entity located within the text input.

17. The computer program product of claim 14, wherein a selected one of the plurality of features is associated with a plurality of optimized statistical models, wherein the computer-usable medium further comprises:

computer-usable program code that determines a value for a first feature in the text input; and

computer-usable program code that selects one of the plurality of optimized statistical models associated with the selected feature according to the value determined for the first feature.

18. The computer program product of claim 14, wherein the computer-usable medium further comprises:

computer-usable program code that compares the complex information target with a plurality of allowable complex information targets; and

computer-usable program code that determines whether the complex information target is valid according to the comparison.

19. The computer program product of claim 18, wherein the computer-usable medium further comprises:

computer-usable program code that selects an alternate value for at least one feature of the complex information target from a plurality of candidate values determined for the at least one feature according to confidence score; and

computer-usable program code that further selects the alternate value from the plurality of candidate values that conforms to an allowable complex information target.

20. The computer program product of claim 14, wherein the computer-usable medium further comprises:

computer-usable program code that compares the complex information target with a list comprising at least one predicted complex information target, wherein each predicted complex information target is associated with an expected complex information target; and

computer-usable program code that, when the complex information target matches a predicted complex information target, maps the complex information target to the expected complex information target associated with the matched predicted complex information target.