Concept matching

• US 20070005343A1
• Filed: 07/01/2005
• Published: 01/04/2007
• Est. Priority Date: 07/01/2005
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Kaplan, Aaron, Sandor, Agnes
• Granted Patent: US 7,689,411 B2
• Time in Patent Office: Days
• US Class Current: 704/9
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method for developing a system for retrieving text related to a selected concept within a text corpus includes identifying a set of semantic classes which express the concept and identifying a set of keywords for each of the semantic classes to be used in text searching in a text corpus. Each set of keywords includes at least one keyword. A plurality of syntactic rules are established which are to be applied to retrieved text which includes keywords. Each of the syntactic rules identifies a first of the semantic classes and a second of the semantic classes. A rule is satisfied when a keyword from the first of the semantic classes is in a syntactic relationship with a keyword from the second of the semantic classes. The syntactic relationship can be any one of a plurality of syntactic relationships.

Claims

1. A method for developing a system for retrieving text related to a selected concept within a text corpus comprising:

identifying a set of semantic classes which express the concept;

identifying a set of keywords for each of the semantic classes to be used in text searching in a text corpus, each set of keywords including at least one keyword;

establishing a plurality of syntactic rules to be applied to retrieved text which includes keywords, each of the syntactic rules identifying a first of the semantic classes and a second of the semantic classes, the rule being satisfied when a keyword from the first of the semantic classes is in a syntactic relationship with a keyword from the second of the semantic classes, the syntactic relationship comprising any one of a plurality of syntactic relationships.

2. The method of claim 1, wherein the method develops a system which identifies text within the text corpus which satisfies at least one of the syntactic rules.

3. The method of claim 1, further comprising:

evaluating the system on a test subset of the text in the text corpus.

4. The method of claim 1, wherein the identifying of the set of semantic classes and the identifying of the set of keywords comprises:

finding sample expressions in text that express the desired concept, and extracting keywords from the sample expressions.

5. The method of claim 1, wherein the identifying of the set of semantic classes and the identifying of the set of keywords comprises:

supplying a keyword or a combination of keywords anticipated to retrieve text expressing the concept to a system which retrieves example text which include the keyword or combination of keywords; and

labeling retrieved examples as relevant or not relevant.

6. The method of claim 1, wherein the identifying of the set of semantic classes and the identifying of the set of keywords comprises:

examining a set of keywords which express a concept and identifying constituent notions that can be combined to express the target concept; and

forming semantic classes of keywords such that all of the keywords in a class express the same constituent notion.

7. The method of claim 6, wherein the examination of the set of keywords and the formation of semantic classes are at least partially performed manually.

8. The method of claim 1 wherein at least one of the semantic classes includes a keyword of another of the semantic classes.

9. The method of claim 1, wherein the establishing of the plurality of syntactic rules comprises:

identifying combinations of semantic classes which can be combined to express the desired concept when syntactically related.

10. The method of claim 1, wherein the establishing of the plurality of syntactic rules comprises:

automatically parsing selected examples of text to identify syntactic relationships; and

manually selecting syntactical rules which select those syntactic relationships which express the desired concept.

11. The method of claim 1, further comprising:

applying a proposed system to a portion of the text corpus; and

manually evaluating the retrieval performance of the system based on the documents retrieved.

12. The method of claim 11, further comprising:

if recall is less than a predetermined minimum recall, modifying at least one of the semantic classes by adding keywords;

if precision is less than a predetermined minimum precision, refining the syntactical rules.

13. The method of claim 1, wherein the syntactic relationships include at least two of the following pairs, where the first of the semantic classes includes a keyword in one of the pair and the second of the semantic classes includes a keyword in the other of the pair:

a verb and its subject;

a verb and its object;

one of an adjective and an adverb and one of a noun, verb and adjective that the adjective or adverb modifies;

a noun and another noun that the noun premodifies; and

a noun and one of a noun and verb that the noun modifies by prepositional attachment.

14. The method of claim 1, wherein there are at least four syntactic relationships.

15. The method of claim 1, wherein each of the syntactic rules applies the same set of syntactic relationships.

16. The method of claim 1, wherein at least one of the semantic classes includes a plurality of keywords.

17. The method of claim 16, wherein at least two of the semantic classes each include at least five keywords.

18. The method of claim 17, wherein at least four of the semantic classes each include at least five keywords.

19. The method of claim 1, wherein there are at least three semantic classes.

20. The method of claim 1, wherein there are fewer than about ten semantic classes.

21. A method for retrieving text related to a selected concept within a text corpus comprising:

identifying a set of semantic classes which, in combinations thereof, express the concept;

identifying a set of keywords for each of the semantic classes to be used in text searching in a text corpus, each set of keywords including at least one keyword;

establishing a plurality of syntactic rules to be applied to retrieved text which includes keywords, each of the syntactic rules identifying a first of the semantic classes and a second of the semantic classes, the rule being satisfied when a keyword from the first of the semantic classes and a keyword from the second of the semantic classes are in any one of a plurality of syntactic relationships; and

applying the syntactic rules to a text corpus to identify text within the text corpus which satisfies at least one of the syntactic rules.

22. A system for developing a text retrieval system comprising:

a memory for storing a set of semantic classes which, in combinations thereof, express a concept;

memory for storing keywords for each of the semantic classes to be used in text searching in a text corpus and for storing syntactic rules to be applied to retrieved text which includes keywords;

a component which suggests sample sentences to a user which include one or more of the stored keywords;

a user input for enabling a user to select sentences from the suggested sentences; and

a component which proposes syntactic rules which are met by the selected sentences.