Labeling of work of art titles in text for natural language processing

• US 20080071519A1
• Filed: 09/19/2006
• Published: 03/20/2008
• Est. Priority Date: 09/19/2006
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Hagege, Caroline, Brun, Caroline
• Granted Patent: US 7,788,084 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

A parser for parsing text includes a tokenizing module which divides the text into an ordered sequence of linguistic tokens. A morphological module associates parts of speech with the linguistic tokens. A detection module identifies candidate titles of creative works, such as works of art. A filtering module filters the candidate titles of works to exclude citations of direct speech from the candidate titles of works. A comparison module compares any remaining candidate titles of works with titles of works in an associated knowledge base. The comparison module annotates the text when a match is found.

Claims

1. A parser for parsing text comprising:

a tokenizing module which divides the text into an ordered sequence of linguistic tokens;

a morphological module for associating parts of speech with the linguistic tokens;

a detection module which applies rules for identifying expressions as candidate titles of works, each of the expressions comprising at least one of the linguistic tokens;

a filtering module for filtering the candidate titles of works, the filtering module applying at least one rule which is formulated to exclude citations of direct speech from the candidate titles of works; and

a comparison module for comparing remaining candidate titles of works with titles of works in an associated knowledge base and annotating the text to identify the candidate title as a nominative unit when a match with a title of a work is found in the associated knowledge base.

2. The parser of claim 1, further comprising an enrichment module which enriches text, for which a match is found, with semantic information derived from information in the knowledge base which is liked to a title of a work.

3. The parser of claim 1, wherein the morphological module identifies verbs which are used to introduce direct speech and wherein the filtering module determines whether an expression identified by the detection module constitutes direct speech introduced by a verb identified by the morphological module as being one of the verbs which are used to introduce direct speech.

4. The parser of claim 1, further comprising a named entity module for identifying person names in the text and wherein the filtering module filters out expressions which are within an identified person name.

5. A natural language processing system comprising:

the parser of claim 1; and

a knowledge base which stores titles of works.

6. The natural language processing system of claim 5, wherein the knowledge base links each title of a work to information which includes at least one of a creator of the work and a category of the work selected from a set of categories.

7. The natural language processing system of claim 5, wherein the parser further comprises an enrichment module which enriches the text with information which includes at least one of the creator of the work and the category of the work selected from a set of categories.

8. A method for natural language processing of input text comprising:

processing the text to identify candidate titles of works;

filtering the candidate titles of works, including applying at least one rule which is formulated to remove citations of direct speech from the candidate titles of works;

comparing remaining candidate titles of works with a knowledge base which identifies titles of works; and

for a candidate title of a work which matches a title of a work in the knowledge base, annotating the text to identify the candidate title of a work as a nominative unit.

9. The method of claim 8, wherein the tiles of works comprise titles of works of art.

10. The method of claim 8, further comprising:

retrieving information from a knowledge base, the information comprising at least one of a category of the matching title of the work and a creator of the matching title of the work; and

enriching the text with information derived from the retrieved information.

11. The method of claim 8, wherein the nominative unit comprises at least one of a noun and a noun phrase.

12. The method of claim 8, wherein the method further comprises identifying verbs which are used to introduce direct speech and wherein the filtering comprises filtering out expressions linked to a verb identified as being one of the verbs which are used to introduce direct speech.

13. The method of claim 8, wherein the filtering further applies a rule to remove at least one of neologisms, quotations included in person names, and nicknames.

14. The method of claim 13, further comprising identifying person names in the text and wherein the filtering comprises filtering out candidate titles of works which are within an identified person name.

15. The method of claim 8, wherein the filtering, processing, comparing, and annotating are all computer implemented.

16. The method of claim 8, wherein the processing includes dividing the text into an ordered sequence of tokens, associating parts of speech with the tokens, and identifying any expressions, each comprising at least one of the tokens, which are distinguished from adjacent tokens in a manner used to identify titles of works.

17. The method of claim 16, wherein the processing includes identifying expressions which are bounded by quotation marks.

18. A storage medium comprising instructions which when executed by a digital processor implement the method of claim 8.

19. A storage medium comprising instructions which when executed by a digital processor implement natural language processing of a text input comprising:

processing the text to identify candidate titles of works;

filtering the candidate titles of works to remove citations of direct speech from the candidate titles of works;

comparing candidate titles of works with a knowledge base which identifies titles of works; and

annotating text which includes a candidate title of a work for which a match is found in the knowledge base.

20. The storage medium of claim 19, wherein the storage medium comprises a tangible storage medium.

21. The storage medium of claim 19, wherein the instructions for annotating the text comprise instructions for identifying the candidate title of the work as a nominative unit.