SYSTEM AND METHOD FOR MINING PATTERNS FROM RELATIONSHIP SEQUENCES EXTRACTED FROM BIG DATA

• US 20140046977A1
• Filed: 01/31/2013
• Published: 02/13/2014
• Est. Priority Date: 08/10/2012
• Status: Abandoned Application

• Current Assignee: Xurmo Technologies Pvt Ltd.
• Original Assignee: Xurmo Technologies Pvt Ltd.
• Inventors: GOPALAKRISHNAN, SRIDHAR, UPADHYAYA, SUJATHA RAVIPRASAD
• Application Number:
  US13/755,047
• Publication Number:
  US 20140046977A1
• Time in Patent Office: Days
• US Class Current: 707/776
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 40/295 Named entity recognition
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation

Abstract

The various embodiments herein provide a system and method for mining frequent patterns in relationship space from a plurality of relationship sequences extracted from a big data The system comprises a data repository for collecting and storing the big data. An Entity Store for collecting and storing a plurality of entities from the big data, an Entity Hierarchy for representing a hierarchical structure of entities, a Relationship Store for collecting and storing relationship instances between the pluralities of entities, a Relationship Hierarchy for representing a hierarchical structure of relationship, a language/domain model for organizing entities and relationships in a hierarchical manner, a pattern query Processing Module (PQPM) for processing, a pattern query related to finding patterns in relationships and entities, and a Pattern Generation Module (PGM) to generate frequent patterns and a Frequent Pattern Display Module (FPDM) to provide a visual presentation of the mined patterns.

Claims

1. A system for mining frequent patterns in relationship space from a plurality of relationship sequences extracted from a big data, the system comprising:

a data repository for collecting and storing the big data;

an entity store for collecting and storing a plurality of entities from the big data;

an entity hierarchy for representing a hierarchical structure of entities;

a relationship store for collecting and storing relationship instances between the plurality of entities from the big data;

a relationship hierarchy for representing a hierarchical structure of relationships;

a language/domain model for organizing entities and relationships in a hierarchical manner;

a pattern query Processing Module (PQPM) for expanding a pattern query related to finding patterns in relationships and entities;

a Pattern Generation Module (PGM) to generate frequent patterns from one or more relationship sequences from the data sources collected based on the pattern query; and

a Frequent Pattern Display Module (FPDM) to provide a visual presentation of the mined patterns;

where the pattern generation module performs frequent pattern mining by extracting relevant relationship sequences from the relationship store using the entity hierarchy and the relationship hierarchy.

2. The system according to claim 1, wherein the big data comprises structured, unstructured and semi-structured data from heterogeneous data sources.

3. The system according to claim 1, wherein the entity store is a collection of entities extracted from the big data, wherein the entity store stores information specific to each entity.

4. The system according to claim 1, wherein the entity hierarchy is a hierarchical structure of entities resolved using Natural Language Processing (NLP) techniques with a support of the language/domain model.

5. The system according to claim 1, wherein the relationship store is adapted to store information related to each relationship instance.

6. The system according to claim 1, wherein the Relationship Hierarchy provides a hierarchical arrangement of relationships by resolving the relationships through at least one of a word-sense disambiguation technique, syntactic and semantic similarity and context resolution technique in conjunction with the language/domain model.

7. The system according to claim 1, wherein the pattern query Processing Module (PQPM) processes the pattern query by expanding, the pattern query in terms of entities after consulting the entity hierarchy, wherein the pattern query is a list comprising entities and relationships of the entities.

8. The system according to claim 1, wherein the pattern query Processing Module (PQPM) performs a expansion of the pattern query to provide a relevant result by disambiguation of the entities in the pattern query, where the disambiguation of the entities in the pattern query is conducted by identifying explicit and implicit similar entities and ignoring the dissimilar entities

9. The system according to claim 1, the Pattern Generation Module (PGM) comprises:

a document retriever to collect documents pertaining to the entities and relationships suggested by the query expansion;

a Relationship Sequence Generator to create a relationship sequence with respect to each of the retrieved documents;

a Frequent Pattern Growth Module (FPGM) for extracting relevant relationship sequences.

10. The system according to claim 9, wherein the Relationship Sequence Generator builds the relationship sequences by treating each relationship as an item, where each relationship sequence comprises the relationships in the order of appearance in the document.

11. The system according to claim 9, wherein the Frequent Pattern Growth Pattern-Mining Module (FPGM) adapts a Frequent Pattern Growth (FPG) algorithm for extracting relevant relationship sequences which considers the relationship sequences as item-sets and extracts the most frequent item-sets.

12. A method for mining frequent patterns from a plurality of relationship sequences extracted from a big data, the method comprising:

extracting a plurality of entities from the big data, where an entity refers to concepts comprising language unit having an independent meaning;

storing the extracted plurality of entities in an entity store;

extracting and storing one or more relationships among the plurality of entities;

building an entity hierarchy by arranging the plurality of entities in a hierarchical manner;

creating a relationship hierarchy by arranging the relationships in a hierarchical manner;

inputting a pattern query, where the pattern query is a list of entities and the relationship of entities;

expanding the pattern query to include most relevant entities and relationships and ignore irrelevant patterns and relationships;

retrieving relevant data sources from data using the pattern query;

building relationship sequences with respect to one or more retrieved data sources;

extracting frequent patterns from the relationship sequences; and

displaying the frequent patterns on a frequent pattern display module.

13. The method according to claim 12, wherein the big data comprises structured, unstructured and semi-structured data from heterogeneous data sources for enabling data analysis on a single view.

14. The method according to claim 12, wherein generating frequent patterns among the relationship sequences is performed using a Frequent Pattern Growth Algorithm which considers the relationship sequences as item-sets and extracts the most frequent item-sets.

15. The method according to claim 12, wherein the method of extracting frequent patterns comprises:

collecting data sources pertaining to one or more entities and relationships contained in a pattern query;

building a relationship sequence pertaining to each of the data source by handling each relationship as an item in an item-set that represents a relationship sequence;

building a relationship sequence in an order the relationships appear in the document; and

identifying the frequent relationship sequences.

16. The method according to claim 12, wherein the method of processing the pattern query comprises:

extracting the hierarchy of the plurality of entities;

expanding the pattern query in terms of entities based on the entity hierarchy; and

expanding the pattern query m terms of relationships based on the relationship hierarchy.

17. The method according to claim 16, expanding the pattern query in terms of entity comprises:

disambiguating the entities the pattern query;

including synonyms and implied entities in the query expansion; and

discarding dissimilar entities.

18. The method according to claim 16, expanding the pattern query in terms of relationships comprises:

resolving relationships according, to the context;

including the relationships which implies context similarity;

including the relationships that are implied within the syntactic similarity; and

discarding the contextually and syntactically dissimilar relationships.