Method and computer system for part-of-speech tagging of incomplete sentences

• US 6,910,004 B2
• Filed: 12/19/2000
• Issued: 06/21/2005
• Est. Priority Date: 12/19/2000
• Status: Expired due to Fees

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Tarbouriech, Nelly, Poirier, Herve
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): HARPER, V PAUL
• Application Number:
  US09/738,987
• Publication Number:
  US 20020077806A1
• Time in Patent Office: 1,645 Days
• Field of Search: 704/4, 704/9, 704/257, 707/4, 715/53.E
• US Class Current: 704/9
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: Y10S 707/99934 Query formulation, input preparation, or translation

Abstract

The invention relates to a method and a computer system for enhanced part-of-speech (POS-) tagging as well as grammatically disambiguating a phrase. A phrase is usually a short multiword expression that may be ambiguous. By introducing grammatical constraints the invention supports POS-tagging as well as grammatically disambiguating the phrase. According to an identifier for the phrase, the phrase is supplemented with artificial context information. The supplemented phrase is then POS-tagged or grammatically disambiguated. Important applications are POS-tagging, Automatic Term Encoding, Headword Detection and Information Retrieval.

Claims

1. A method, for use in a computer system, for assigning at least one part-of-speech (POS) tag to a phrase, the method comprising:

obtaining an identifier for the phrase, the identifier being associated with context information;

supplementing the phrase with the context information; and

assigning the at least one POS-tag to the phrase based on the supplemented phrase.

2. The method of claim 1, wherein the context information comprises at least one of pre-context and post-context information.

3. The method of claim 1, wherein the identifier is a main grammatical category of the phrase on the whole.

4. The method of claim 1, wherein the identifier is defined by a structural property of the phrase in a plurality of phrases or textual information.

5. The method of claim 1, further comprising deriving a formal structure for the phrase, that covers variations of the phrase.

6. The method of claim 1, further comprising extracting a headword out of the phrase based on the phrase with the at least one assigned POS-tag.

7. The method of claim 1, wherein the step of obtaining the identifier further comprises selecting a main grammatical category of the phrase on the whole according to the identifier, the main grammatical category being associated to the context information to be supplemented to the phrase.

8. The method of claim 1, wherein the step of obtaining the identifier further comprises:

preselecting probable main grammatical categories for the phrase according to the identifier; and

selecting a most probable category from the preselected main grammatical categories, the most probable category being associated to the context information to be supplemented to the phrase.

9. The method of claim 8, wherein the step of selecting the most probable category comprises obtaining an external selection of the most probable category.

10. The method of claim 8, wherein the at least one POS-tag assigned to the phrase is selected from potential POS-tags assignable to the phrase without context information;

and selecting the most probable category is supported by evaluating the potential POS-tags.

11. The method of claim 1, wherein the context information includes at least one POS-tag.

12. The method of claim 1, wherein the context information includes textual information.

13. The method of claim 1, wherein the computer system comprises:

a primary storage comprising;

a computer program comprising processor-executable instructions implementing;

a context supplementer for supplementing the context information to the phrase; and

a POS-tagger for assigning the at least one tag to the phrase;

a context storage comprising a plurality of context information items; and

an identifier storage comprising a plurality of identifiers;

each of which being associated with at least one context information item of the plurality of context information items;

identifier input means for obtaining the identifier for the phrase, the identifier being associated to the context information according to the plurality of identifiers and its association to the plurality of context information items; and

a processor, connected to the primary storage and the identifier input means, for executing the processor executable instructions.

14. A method, for use in a computer system, for grammatically disambiguating a phrase, the method comprising:

acquiring the phrase;

acquiring an identifier for the phrase, which is associated with artificial information, the artificial information supporting grammatical disambiguation of the phrase;

supplementing the phrase with the artificial information; and

grammatically disambiguating the phrase based on the supplemented phrase.

15. A computer system for part-of-speech (POS)-tagging of a phrase, the system comprising:

identifier input means for obtaining an identifier for the phrase, the identifier being associated to context information;

a context storage comprising a plurality of context information items;

an identifier storage, connected to the context storage, comprising a plurality of identifiers;

each of which being associated with at least one context information item of the plurality of context information items;

a context supplementer, connected to the identifier input and the context storage, for supplementing the phrase with the associated context information of the obtained identifier; and

a POS-tagger, connected to the context supplementer, for identifying the part of speech of each part of the phrase based on the supplemented phrase.

16. The system of claim 15, further comprising phrase input means, connected to the context supplementer, for getting the phrase.

17. The system of claim 15, further comprising:

a category storage, connected to the identifier storage, for storing a plurality of categories, the identifier being associated with at least one of the plurality of categories; and

a category evaluator, connected to the category storage, the identifier storage and the context supplementer, for evaluating the category of the phrase based on the identifier.

18. The system of claim 15, further comprising category input means connected to the context supplementer for obtaining a category associated to one of the context information items, which is supplemented to the phrase.

19. The system of claim 15, further comprising output means connected to the POS-tagger for outputting the at least one POS-tag.