Systems and methods for information extraction using contextual pattern discovery

• US 8,630,989 B2
• Filed: 05/27/2011
• Issued: 01/14/2014
• Est. Priority Date: 05/27/2011
• Status: Active Grant

• Current Assignee: Beijing Zitiao Network Technology Company Limited (ByteDance Ltd.)
• Original Assignee: International Business Machines Corporation
• Inventors: Blohm, Sebastian Johannes, Chu, Vivian Yaw-Wen, Ho, Ching-Tien, Li, Yunyao, Zhu, Huaiyu
• Primary Examiner(s): NGUYEN, KIM T
• Application Number:
  US13/117,570
• Publication Number:
  US 20120303661A1
• Time in Patent Office: 963 Days
• Field of Search: 707/705, 707/776
• US Class Current: 707/705
• CPC Class Codes: G06F 40/295 Named entity recognition

Abstract

Described herein are methods, systems, apparatuses and products for automatically discovering patterns in a text corpus. An aspect provides extracting at least one context string related to at least one annotator from the at least one text corpus; analyzing the at least one context string for at least one sequence, the at least one sequence comprised of at least one subsequence; determining at least one sequence signature for each at least one sequence by applying applicable rules to the at least one sequence; and grouping the at least one sequence signature into at least one group.

Claims

1. A system comprising:

at least one processor; and

a memory device operatively connected to the at least one processor;

wherein, responsive to execution of program instructions accessible to the at least one processor and configured to automatically discover at least one text-based pattern in at least one text corpus, the at least one processor is configured to;

issue a query of the text corpus to extract at least one context string comprising a sequence of text from the text corpus, the sequence of text identified using a positional relationship to at least one text annotator corresponding to a text entity of interest included in text of the at least one text corpus;

analyze the at least one context string to produce at least one sequence representing a text-based pattern of the context string;

determine at least one semantic sequence signature for the context string from the at least one sequence which identifies the context string; and

thereupon use the at least one semantic sequence signature to automatically group semantically similar context strings of the text corpus.

2. The system according to claim 1, wherein extracting at least one context string comprises querying the at least one text corpus using at least one structured query.

3. The system according to claim 2, wherein the at least one structured query comprises at least one annotator and at least one context condition for defining positional context of text of the text corpus with respect to the at least one annotator.

4. The system of claim 1, wherein semantically similar context strings share a common semantic sequence signature.

5. The system of claim 4, wherein, for a context string having more than one sequence, identifying the semantic signature for the context string from among the more than one sequence.

6. The system of claim 5, wherein the at least one processor is configured to output, for a plurality of semantically similar grouped context strings, a structured document comprising an original text context field and a semantic sequence signature field.

7. The system of claim 1, wherein the text corpus comprises one of unstructured text and semi-structured text.

8. A method comprising:

automatically discovering at least one text-based pattern in at least one text corpus, wherein discovering at least one pattern comprises;

issuing a query of the text corpus to extract at least one context string comprising a sequence of text from the text corpus, the sequence of text identified using a positional relationship to at least one annotator corresponding to a text entity of interest included in text of the at least one text corpus;

analyzing the at least one context string to produce at least one sequence representing a text-based pattern of the context string;

determining at least one semantic sequence signature for the context string from the at least one sequence which identifies the context string; and

thereupon using the at least one semantic sequence signature to automatically group semantically similar context strings of the text corpus.

9. The method according to claim 8, wherein:

extracting at least one context string comprises querying the at least one text corpus using at least one structured query; and

the at least one text corpus is organized as a trie data structure.

10. A computer program product comprising:

a computer readable storage medium having computer readable program code configured to automatically discover at least one text-based pattern in at least one text corpus embodied therewith, the computer readable program code comprising;

computer readable program code configured to issue a query of the text corpus to extract at least one context string comprising a sequence of text from the text corpus, the sequence of text identified using a positional relationship to at least one annotator corresponding to a text entity of interest included in text of the at least one text corpus;

computer readable program code configured to analyze the at least one context string to produce at least one sequence representing a text-based pattern of the context string;

computer readable program code configured to determine at least one semantic sequence signature for the context string from the at least one sequence which identifies the context string; and

computer readable program code configured to thereupon using the at least one semantic sequence signature to automatically group semantically similar context strings of the text corpus.