CONCEPTUAL DOCUMENT ANALYSIS AND CHARACTERIZATION

• US 20160328454A1
• Filed: 07/20/2016
• Published: 11/10/2016
• Est. Priority Date: 04/27/2015
• Status: Active Grant

• Current Assignee: Consilio LLC
• Original Assignee: Altep, Inc. (Consilio LLC)
• Inventors: Miller, Roger W., van den Berge, Willem R.
• Granted Patent: US 9,886,488 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

Data files are received from data sources that include textual content. The data files are categorized using a taxonomy of categories, where each category has sample textual content that defines a concept for the category. The categorizing includes comparing the textual content of the data file with the sample textual content for the category. A file score is calculated for each data file to compare the degree of similarity between the defined concept of the category and a determined concept for the data file. Each data file is associated with the category if the file score is equal to or greater than a pre-determined minimum score for the category. A portion of the data file and/or file score is be provided.

Claims

1. A method comprising:

receiving a plurality of data files from a plurality of data sources that comprise textual content;

categorizing the plurality of data files using a taxonomy of categories in which each category has associated sample textual content defining a concept for the category, the categorizing comprising, for each category;

comparing, for each of the plurality of data files, the textual content of the data file with the sample textual content for the category;

calculating, based on the comparing and for each of the plurality of data files, a file score corresponding to the degree of similarity between the defined concept of the category and a determined concept for the data file; and

associating, for each of the plurality of data files, the data file with the category if the file score is equal to or greater than a pre-determined minimum score for the category; and

providing at least a portion of the data file and/or the associated file score.

2. The method of claim 1, further comprising:

generating the taxonomy by;

adding, to the taxonomy, at least one of the categories, with each of the at least one categories representing the first concept;

adding, to the at least one category, a sample comprising the sample textual content corresponding to the first concept; and

adding, to the at least one category, the minimum score.

3. The method of claim 1, wherein the associating is between the data file and only one category, the category being the category generating the highest file score equal to or greater than the minimum score.

4. The method of claim 1 further comprising clustering the textual content into at least one cluster, the cluster representative of identified concepts.

5. The method of claim 1 wherein the data file further comprises a source identifier identifying the data source.

6. The method of claim 1, wherein at least one text item is identified from the sample textual content and the text item is given a text item score to identify the relevance of the text item to the sample.

7. The method of claim 1, wherein the providing includes providing a first representation of the data file along with a second representation of all attachments, metadata, or electronic associations.

8. The method of claim 1, wherein providing at least a portion of the data file and/or the associated file score comprises at least one of:

displaying at least a portion of the data file and/or the associated file score, loading at least a portion of the data file and/or the associated file score into memory, transmitting data including at least a portion of the data file and/or the associated file score to a remote computing device, or storing at least a portion of the data file and/or the associated file score into persistent memory.

9. The method of claim 8, wherein the persistent memory is at least one of:

a local memory, cloud memory, or e-mail servers.

10. The method of claim 1, wherein the receiving is from an ongoing data stream providing an ongoing source of the data files to be categorized.

11. The method of claim 10, wherein the ongoing data stream is from an e-mail server.

12. A non-transitory computer program product storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to perform operations comprising:

receiving a plurality of data files from a plurality of data sources that comprise textual content;

categorizing the plurality of data files using a taxonomy of categories in which each category has associated sample textual content defining a concept for the category, the categorizing comprising, for each category;

comparing, for each of the plurality of data files, the textual content of the data file with the sample textual content for the category;

calculating, based on the comparing and for each of the plurality of data files, a file score corresponding to the degree of similarity between the defined concept of the category and a determined concept for the data file; and

associating, for each of the plurality of data files, the data file with the category if the file score is equal to or greater than a pre-determined minimum score for the category; and

providing at least a portion of the data file and/or the associated file score.

13. The computer program product of claim 12, wherein the operations of generating the taxonomy further comprise:

adding, to the taxonomy, at least one of the categories, with each of the at least one categories representing the first concept;

adding, to the at least one category, a sample comprising the sample textual content corresponding to the first concept; and

adding, to the at least one category, the minimum score.

14. The computer program product of claim 12, wherein the operations further comprise clustering the textual content into at least one cluster, the cluster representative of identified concepts.

15. The computer program product of claim 12, wherein at least one text item is identified from the sample textual content and the text item is given a text item score to identify the relevance of the text item to the sample.

16. The computer program product of claim 12, wherein the operations of providing includes providing a first representation of the data file along with a second representation of all attachments, metadata, or electronic associations.

17. A system comprising:

at least one programmable data processor; and

memory storing instructions which, when executed by the at least one programmable data processor, result in operations comprising;

receiving a plurality of data files from a plurality of data sources that comprise textual content;

categorizing the plurality of data files using a taxonomy of categories in which each category has associated sample textual content defining a concept for the category, the categorizing comprising, for each category;

comparing, for each of the plurality of data files, the textual content of the data file with the sample textual content for the category;

calculating, based on the comparing and for each of the plurality of data files, a file score corresponding to the degree of similarity between the defined concept of the category and a determined concept for the data file; and

associating, for each of the plurality of data files, the data file with the category if the file score is equal to or greater than a pre-determined minimum score for the category; and

providing at least a portion of the data file and/or the associated file score.

18. The system of claim 17, wherein the operations of generating the taxonomy further comprise:

generating the taxonomy by;

adding, to the taxonomy, at least one of the categories, with each of the at least one categories representing the first concept;

adding, to the at least one category, a sample comprising the sample textual content corresponding to the first concept; and

adding, to the at least one category, the minimum score.

19. The system of claim 17, wherein the operations further comprise clustering the textual content into at least one cluster, the cluster representative of identified concepts.

20. The system of claim 17, wherein the operations of providing further comprise providing a first representation of the data file along with a second representation of all attachments, metadata, or electronic associations.