System and method for identifying base noun phrases

• US 6,859,771 B2
• Filed: 06/04/2001
• Issued: 02/22/2005
• Est. Priority Date: 04/23/2001
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhou, Ming, Huang, Chang-Ning, Xun, Endong
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): SPOONER, LAMONT M
• Application Number:
  US09/873,656
• Publication Number:
  US 20030014238A1
• Time in Patent Office: 1,359 Days
• Field of Search: 704/1, 704/4, 704/9, 704/10
• US Class Current: 704/1
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A system and method identify base noun phrases (baseNP) in a linguistic input. A part-of-speech tagger identifies N-best part-of-speech tag sequences corresponding to the linguistic input. A baseNP identifier identifies baseNPs in the linguistic input using a unified statistical model that identifies the baseNPs, given the N-best POS sequences.

Claims

1. A method of processing a natural language input, comprising:

identifying N-best part-of-speech POS) sequences corresponding to the natural language input;

identifying a likely base noun phrase (baseNP) sequence based on the N-best POS sequences identified; and

outputting the likely baseNP sequence.

2. The method of claim 1 wherein identifying a likely baseNP sequence comprises:

identifying a plurality of baseNP sequences for each of the N-best POS tag sequences; and

calculating which of the plurality of baseNP sequences is most likely.

3. The method of claim 2 wherein calculating which of the plurality of baseNP sequences is most likely comprises:

calculating a likely baseNP sequence that is most likely based on lexical information indicative of a position of words in the natural language input relative to baseNPs identified in the baseNP sequences.

4. The method of claim 3 wherein calculating a likely baseNP sequence that is most likely based on lexical information comprises:

calculating a likely baseNP sequence that is most likely based on lexical information indicative of POS tags assigned to the words in the natural language input.

5. The method of claim 3 wherein calculating a likely baseNP sequence comprises:

calculating a likely baseNP sequence based on the lexical information for every word in the natural language input.

6. The method of claim 2 wherein the natural language input comprises a sentence and wherein calculating which of the plurality of baseNP sequences is most likely comprises:

calculating which of the plurality of baseNP sequences is most likely over the entire sentence.

7. The method of claim 3 wherein a baseNP rule comprises a sequence of POS tags corresponding to words in the natural language input identified as a baseNP and wherein calculating a likely baseNP sequence comprises:

calculating a probability of POS tags and baseNP rules, given their context.

8. The method of claim 7 wherein calculating a probability of POS tags and baseNP rules comprises:

calculating the probability of POS tags and baseNP rules given n prior POS tags or baseNP rules.

9. The method of claim 1 wherein identifying a likely baseNP sequence includes:

calculating a probability of each of the N-best POS sequences given the natural language input.

10. A natural language processing system for processing a natural language input, comprising:

a base noun phrase (baseNP) identifier configured to receive N-best part-of-speech (POS) tag sequences for the natural language input and identify a likely baseNP sequence of baseNPs corresponding to the natural language input, given the N-best POS tag sequences a POS tagger, coupled to the baseNP identifier, receiving the natural language input and calculating the N-Best POS tag sequences corresponding to the natural language input.

11. The system of claim 10 wherein the baseNP identifier is configured to identify a plurality of baseNP sequences for each of the POS tag sequences and calculate which of the plurality of baseNP sequences is the likely baseNP sequence.

12. The system of claim 11 wherein the baseNP identifier further comprises:

a unified statistical model that includes lexical information indicative of a position of words in the natural language input relative to baseNPs identified in the baseNP sequences.

13. The system of claim 12 wherein a baseNP rule comprises a sequence of POS tags corresponding to words in the natural language input that are identified as a baseNP and wherein the unified statistical model includes a baseNP rule component for calculating a probability of POS tags and baseNP rules, given contextual information.

14. The system of claim 13 wherein the baseNP rule component is configured to calculate the probability of POS tags and baseNP rules, given n prior POS tags and baseNP rules.

15. The system of claim 14 wherein the natural language input comprises a sentence and wherein the unified statistical model is configured for calculating which of the plurality of baseNP sequences is most likely over the entire sentence.

16. A method of processing a linguistic input, comprising:

identifying N-best part-of-speech (POS) sequences corresponding to the linguistic input;

identifying one or more base noun phrases (baseNPs) for each of the N-best POS sequences to form a plurality of different possible baseNP sequences corresponding to each of the POS sequences;

for each baseNP sequence, identifying whether it is a likely baseNP sequence based on a probability of the associated POS sequence and a probability of the baseNP sequence, given lexical information indicative of a position of words in the linguistic input relative to the baseNPs identified in the baseNP sequence; and

outputting the likely baseNP sequence identified.

17. The method of claim 16 wherein identifying one or more baseNPs for each of the N-best POS sequences comprises:

identifying baseNP rules for each of the N-best POS sequences, the baseNP rules comprising a sequence of POS tags corresponding to words in the linguistic input identified as a baseNP.

18. The method of claim 17 wherein identifying whether each baseNP sequence is a likely baseNP sequence, comprises:

calculating a probability of POS tags and baseNP rules, given n prior POS tags or baseNP rules in the POS sequence.

19. The method of claim 18 wherein the linguistic input comprises a sentence and wherein identifying whether each baseNP sequence is a likely baseNP sequence comprises:

identifying whether each baseNp sequence is a likely baseNP sequence over the entire sentence.