System and method for mining data

• US 7,533,069 B2
• Filed: 06/16/2006
• Issued: 05/12/2009
• Est. Priority Date: 02/01/2002
• Status: Active Grant

• Current Assignee: John Fairweather
• Original Assignee: John Fairweather
• Inventors: Fairweather, John
• Primary Examiner(s): STARKS, WILBERT L
• Application Number:
  US11/455,304
• Publication Number:
  US 20060235811A1
• Time in Patent Office: 1,061 Days
• Field of Search: 706/12, 706/45, 706/47, 704/9
• US Class Current: 706/12
• CPC Class Codes: G06F 8/427 Parsing
• CPC Class Codes: G06F 9/4493 Object persistence
• CPC Class Codes: G06K 13/0825 the ejection arrangement being of the push-push kind
• CPC Class Codes: Y10S 707/913 Multimedia
• CPC Class Codes: Y10S 707/966 Distributed
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A system and method for extracting data, hereinafter referred to as MitoMine, that produces a strongly-typed ontology defined collection referencing (and cross referencing) all extracted records. The input to the mining process can be any data source, such as a text file delimited into a set of possibly dissimilar records. MitoMine contains parser routines and post processing functions, known as ‘munchers’. The parser routines can be accessed either via a batch mining process or as part of a running server process connected to a live source. Munchers can be registered on a per data-source basis in order to process the records produced, possibly writing them to an external database and/or a set of servers. The present invention also embeds an interpreted ontology based language within a compiler/interpreter (for the source format) such that the statements of the embedded language are executed as a result of the source compiler ‘recognizing’ a given construct within the source and extracting the corresponding source content. In this way, the execution of the statements in the embedded program will occur in a sequence that is dictated wholly by the source content. This system and method therefore make it possible to bulk extract free-form data from such sources as CD-ROMs, the web etc. and have the resultant structured data loaded into an ontology based system.

Claims

1. A data mining method in a data processing system comprising a processor and a memory for storing data mined from a variety of sources into a structured target data model, the method comprising the steps of:

(a) receiving a first source data for mining by the data processing system;

(b) parsing said first source data by the data processing system;

said parsing step further comprising;

(b1) parsing a source format of said first source data with an outer parser running in the processor;

(b2) based on said parsed source format, processing selected data extracted from said first source data with an inner level parser embedded in said outer parser; and

(b3) executing by said inner level parser one or more statements in an order dictated by a content of said first source data as dictated by an evolution of a parsing state of said outer level parser, wherein said one or more statements are expressed using an ontology description language having one or more data types and one or more data fields which are directly manipulated and assigned by said one or more statements executed within said inner level parser through said ontology description language without explicit declarations therein;

(c) creating, as a result of said parsing steps, a first collection of records conformed to the structured target data model as described by said ontology description language, wherein each of said records in said first collection of records are referenced and cross-referenced to each other;

(d) storing said first collection of records conformed to the structured target data model in the memory;

(e) retrieving said first collection of records for further processing by the data processing system.

2. The method according to claim 1 further comprising the steps of:

receiving a next source data for mining by the data processing system; and

repeating steps (b) through (d) for said next source data, wherein a next collection of records are referenced and cross-referenced to each other and to said first collection of records.

3. The method according to claim 1, wherein said parsing step further comprises the step of:

utilizing a named source specific lexical analyzer specification for parsing said first source data.

4. The method according to claim 1, wherein said parsing step further comprises the step of:

utilizing a named source specific Backus-Naur Form (BNF) specification for parsing said first source data.

5. The method according to claim 4, wherein said utilizing step further comprises the step of:

utilizing one or more embedded plug-ins specified in said named source specific BNF specification to invoke and supply said one or more statements to said inner level parser.

6. The method according to claim 1 further comprising the step of:

altering said first data source before said parsing step.

7. The method according to claim 1 further comprising the steps of:

registering one or more post processing functions; and

executing said one or more post processing functions on said first collection of records prior to said storing step.

8. The method according to claim 1 further comprising the step of:

querying said first collection of records stored in the memory for cross-source analysis.

9. The method according to claim 1 further comprising the step of:

performing said parsing step via a batch mining process.

10. The method according to claim 1 further comprising the step of:

performing said parsing step as part of a running server process connected to a live source for said first source data.

11. A data processing system for mining data from a variety of sources for storing in a structured target data model, the system comprising:

a processor;

an outer parser running in said processor, wherein said outer parser parses a source format of a first source data received by said processor;

an inner level parser embedded in said outer parser for parsing said first data source based on said parsed source format;

an ontology description language in which the structured target data model is specified and which is also utilized by said inner level parser, wherein said inner level parser executes one or more statements in an order dictated by a content of said first source data, wherein said one or more statements are expressed in said ontology description language and access one or more data types and one or more data fields that are directly manipulated and assigned within said inner level parser using said ontology description language without explicit declarations therein; and

a memory for storing a first collection of records created by said outer parser and said inner level parser, wherein said first collection of records conform to the structured target data model, and each of said records in said first collection of records are referenced and cross-referenced to each other, wherein said first collection of records are retrieved from the memory for further processing by the data processing system.

12. The system according to claim 11 further comprising:

a named source specific lexical analyzer specification passed to said outer level parser for use in tokenizing said first source data.

13. The system according to claim 11 further comprising:

a named source specific Backus-Naur Form (BNF) specification passed to said outer level parser for use in parsing said first source data.

14. The system according to claim 13 further comprising:

one or more embedded plug-ins specified in said named source specific BNF specification for invoking and supplying said one or more statements to said inner parser.

15. The system according to claim 14 wherein said named source specific BNF specification further comprises:

a specification for said one or more embedded plug-ins that are passed textual strings consisting of interpretable source statements.

16. The system according to claim 15 wherein said one or more embedded plug-ins pass said one or more statements to said inner parser via said textual strings thereby determining said order of execution of said one or more statements in said inner parser based on a parser state of said outer parser as determined by said contents of said first source data.

17. The system according to claim 11 further comprising:

a line processing function running in said processor, wherein said line processing function permits pre-examination and alteration of lines appearing in said first source data before said first source data is processed by said outer parser.

18. The system according to claim 11 further comprising:

one or more post processing functions, wherein said one or more post processing functions are executed on said first collection of records prior to storing in said memory.

19. The system according to claim 11 further comprising:

a set of assignable and readable registers logically associated with each instance of said inner level parser and said outer parser, wherein said registers are used to store any data assigned to them or to store a current state of said outer parser and said inner level parser, and which may be accessed and manipulated by said one or more statements executed by said inner level parser.

20. The system according to claim 11 further comprising:

an application user interface for invoking the system to mine data from said first data source.

21. The system according to claim 11 further comprising:

a server running a server process, wherein the system can be registered with said running server process connected to said first data source in order to allow said server to mine data from said first data source on a continuous unattended basis.