Method and System for Creating a Data Profile Engine, Tool Creation Engines and Product Interfaces for Identifying and Analyzing Files and Sections of Files

• US 20100191748A1
• Filed: 09/15/2009
• Published: 07/29/2010
• Est. Priority Date: 09/15/2008
• Status: Active Grant

• Current Assignee: Kmstandards LLC
• Original Assignee: Kiiac LLC
• Inventors: Martin, Kingsley, Liggett, Tracy S.
• Granted Patent: US 8,606,796 B2
• Time in Patent Office: Days
• US Class Current: 707/750
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

A data profile engine identifies, classifies, analyzes, searches, compares and cross-references entire files and sections of files, records and other forms of electronic media, and a tool creation engine in combination with the data profile engine builds custom solutions and product interfaces.

Claims

1. A method for identifying a relationship between a plurality of data files comprising word text, the method comprising:

receiving at a processor the plurality of data files from one or more computer databases;

deconstructing the data files into one or more text blocks;

creating a data profile for each data file and for the one or more text blocks associated with each data file, each data profile comprising a statistical signature for a set of data forming the corresponding data file or text block compared to the plurality of data files; and

storing the data profiles on the one or more computer databases.

2. The method of claim 1, wherein the processor is further configured to match the text blocks within each file and across the plurality of data files based on the data profiles to create a plurality of text block groups.

3. The method of claim 2, wherein the processor:

generates a data profile for each of the plurality of text block groups and compares each of the text block group data profiles to determine matching characteristics in order to identify text block group sets; and

based on the text block groups, generates a data profile for each of the plurality of text block group sets.

4. The method of claim 3, wherein the processor is further configured to successively refine the data profile for each text block, text block group, and text block group set by applying the profile data generated for each of the text blocks, text block groups, and text block group sets to the data profiles of the other text blocks, text block groups, and text block group sets.

5. The method of claim 2, wherein the processor is further configured for:

generating a word universe comprising word frequencies, word weights, and statistical information for words in the plurality of files; and

wherein creating the data profile for each data file comprises measuring word commonality, mean and divergence among the words in the each of the data files compared to the word universe and relationships between and among text blocks and text block groups.

6. The method of claim 5, wherein creating the data profile further comprises calculating a base score for each text block according to a word weight sum, the word weight sum calculated based on a statistical frequency of each word in the text block compared to the word universe.

7. The method of claim 6, wherein the processor is further configured to:

match the text blocks within each file and across the plurality of data files based on comparing the base scores in each of the text block data profiles in order to create a plurality of text block groups; and

generate a data profile for each of the plurality of text block groups, the data profile comprising word characteristics and text block match statistics.

8. The method of claim 7, wherein the text block group data profile comprises statistical data related to hierarchical relationships between and among text blocks and text block groups.

9. The method of claim 7, wherein the processor:

generates a data profile for each of the plurality of text block groups and compares each of the text block group data profiles to determine matching characteristics in order to identify text block group sets; and

based on the text block groups, generates a data profile for each of the plurality of text block group sets.

10. The method of claim 9, wherein the processor is further configured to successively refine the data profile for each text block, text block group, and text block group set by applying the profile data generated for each of the text blocks, text block groups, and text block group sets to the data profiles of the other text blocks, text block groups, and text block group sets.

11. The method of claim 1, wherein deconstructing the data files comprises analyzing lexical characteristics of the text including identifying paragraph breaks and identification of headings based on text formatting and using information in the data profiles to identify text block delimitation.

12. A system for analyzing a document comprising text, the system comprising:

a processor configured to receive the document and perform the steps of;

deconstructing the document into one or more text blocks;

creating a data profile for each of the one or more text blocks, each data profile comprising a statistical signature for a set of data forming the text block; and

comparing the data profile for each of the one or more text blocks with a template stored on a computer database, the template comprising data profiles for matching text blocks from a source set of documents; and

a user interface coupled to the processor for displaying an indication of similarity of the document compared to the template, the indication of similarity comprising statistical measure of frequency for matching text blocks.

13. The system of claim 12, wherein the indication of similarity further comprises a measure of language commonality compared to a text block data profile associated with the template.

14. The system of claim 12, wherein the indication of similarity further comprises a measure of language conformity compared to a text block data profile associated with the template.

15. The system of claim 12, further comprising providing an indication of divergence of the document compared to the template, the indication of divergence comprising a list of text blocks missing from the selected document compared to the template.

16. The system of claim 15, wherein the indication of divergence further comprises a measure of language variance compared to a text block data profile associated with the template.

17. A system for preparing a document comprising text, the system comprising:

a processor configured to receive the document and perform the steps of;

deconstructing the document into one or more text blocks;

creating a data profile for each of the one or more text blocks, each data profile comprising a statistical signature of a set of data forming the text block; and

comparing the data profile for each of the one or more text blocks with data profiles associated with a model document stored on a computer database, the model document comprising a plurality of statistically similar text blocks from a source set of model documents; and

a user interface coupled to the processor for displaying default standard clauses, alternative clauses and infrequently used clauses based on the source set of model documents.

18. The system of claim 17, wherein the processor is further configured to generate a prototypical document providing the standard clauses, the alternative clauses and the infrequently used clauses in a redline format.

19. The system of claim 17, wherein the system is further configured to refine the model document and its associated profile based on the received document.

20. The system of claim 17, wherein the system is further configured for providing an exemplar text block from a set of matching text blocks that most closely conforms to the data profile of one of the one or more text blocks from the received document.

21. The system of claim 20, further comprising providing alternative matching text blocks from the set of matching text blocks based on statistically similar data profiles compared to the data profile of the one of the one or more text blocks from the received document.

22. A system for searching entire files and sections of files, the system comprising:

a processor configured to receive a plurality of documents and perform the steps of;

deconstructing the plurality of documents into one or more text blocks;

creating a data profile for each of the one or more text blocks, each data profile comprising a statistical signature of a set of data forming the text block; and

comparing the data profile for each of the one or more text blocks with data profiles associated with a model document stored on a computer database, the model document comprising a plurality of statistically similar text blocks from a source set of model documents;

a user interface coupled to the processor for entering a search, the search comprising search terms, sections captions and/or text of a similar section of a user document compared to the model document; and

a display for displaying the search results.

23. The system of claim 22, wherein the system is further configured for:

finding matching sections of the model document using one or more sections of the user document as a search pattern by comparing the one or more sections and a source file from which the one or more sections originates to other data profiles associated with the model document.

24. The system of claim 22, wherein the system is further configured to categorize the user document based on analyzing and comparing data profiles associated with the model document with the user document or one or more sections of the user document.

25. The system of claim 22, wherein the system is further configured for categorizing topical sections the user document based on the processor deconstructing the user document, creating a data profile for the user profile, and comparing the user document data profiles with the data profiles associated with the model document.