Policy based data collection, processing, and negotiation for analytics

• US 10,223,329 B2
• Filed: 03/20/2015
• Issued: 03/05/2019
• Est. Priority Date: 03/20/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Baset, Salman A., Deng, Yu, Kim, Minkyong, Segal, Alla, Schulz, Charles O., Sailer, Anca, Wang, Long
• Primary Examiner(s): Thomas, Ashish
• Assistant Examiner(s): Ferrer, Jedidiah P
• Application Number:
  US14/664,225
• Publication Number:
  US 20160275158A1
• Time in Patent Office: 1,446 Days
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/367 Ontology

Abstract

An approach for ontological policy based data collection, processing, and negotiation for data in view of analytics is provided. The approach searches one or more data sources for data related to a data request. The approach collects data related to the data request from the one or more data sources. The approach determines whether one or more attributes generated from the data request match one or more descriptors associated with the data related to the data request. The approach creates one or more annotated ontologies for the data related to the data request. The approach displays a hierarchical visualization of the one or more annotated ontologies for the data related to the data request. The approach updates the one or more annotated ontologies for the data related to the data request based, at least in part, on an evaluation of the quality of the one or more data selections.

Claims

1. A method for improving data collection and analytics utilizing off-line data auto-tagging and on-line fine granularity data tagging based on negotiations between a data provider and a data consumer, the method comprising:

receiving, by one or more computer processors, a data request from the data consumer;

generating, by one or more computer processors, one or more attributes from the data request;

searching, by one or more computer processors, one or more data sources for data related to the one or more attributes;

collecting, by one or more computer processors, data related to the one or more attributes from the one or more data sources, wherein collecting the data related to the one or more attributes includes populating a data descriptor template for the data related to the one or more attributes with one or more descriptors;

determining, by one or more computer processors, whether the collected data match the one or more descriptors;

responsive to a determination that the collected data match the one or more descriptors, creating, by one or more computer processors, one or more annotated ontologies for the collected data by combining one or more ontologies defining a domain for the collected data with one or more modifiable data policy rules to describe the data and one or more handling requirements for the collected data based on the one or more modifiable data policy rules, wherein creating the one or more annotated ontologies includes annotating the one or more descriptors in the data descriptor template with at least one summary of one or more documents from the collected data, wherein annotating the one or more descriptors with the at least one summary includes associating summary information generated from searching the one or more data sources, applying data policy rules to the at least one summary, and modifying the at least one summary to conform to relevant data policy rules related to data cost, data topology and quantity of data;

providing, by one or more computer processors, a hierarchical visualization of the one or more annotated ontologies for the collected data to the data consumer;

receiving, by one or more computer processors, one or more data selections from the one or more annotated ontologies; and

updating, by one or more computer processors, the one or more annotated ontologies for the collected data based, at least in part, on a quality evaluation of the one or more data selections.

2. The method of claim 1, wherein collecting the data related to the one or more attributes, further comprises:

extracting, by one or more computer processors, the one or more descriptors from the data related to the one or more attributes, wherein the one or more descriptors describe and identify the data; and

determining, by one or more computer processors, one or more ontologies defining a domain for the data related to the one or more attributes.

3. The method of claim 2, wherein determining one or more ontologies defining a domain for the data related to the one or more attributes, further comprises:

discovering, by one or more computer processors, one or more relevant ontology descriptors; and

associating, by one or more computer processors, the one or more relevant ontology descriptors with one or more descriptors from the data related to the one or more attributes.

4. The method of claim 1, wherein determining whether the collected data match one or more descriptors, further comprises at least one of:

determining, by one or more computer processors, that the one or more attributes generated from the data request match the one or more descriptors associated with the collected data where at least one of one or more features of the collected data are related to at least one of the one or more attributes generated from the data request; and

determining, by one or more computer processors, that the one or more attributes generated from the data request do not match the one or more descriptors associated with the collected data where at least one of one or more features of the collected data are not related to at least one of the one or more attributes generated from the data request.

5. The method of claim 1, wherein providing a hierarchical visualization of one or more ontologies for the collected data, further comprises:

associating, by one or more computer processors, one or more user defined policy rules with one or more descriptors from the collected data, wherein associating the one or more user defined policy rules with one or more descriptors for the collected data includes determining a level of relevancy between policy rules and data descriptors.

6. The method of claim 1, further comprises:

filtering, by one or more computer processors, data from the one or more ontologies for the collected data, at least in part, on the one or more data selections, a data consumer data policy, and a data provider data policy, wherein filtering includes cleaning data from the one or more ontologies by applying one or more modifiable data policy rules from the data consumer data policy and the data provider data policy.

7. The method of claim 1, wherein updating the one or more ontologies for the collected data based, at least in part, on a quality evaluation of the quality of the one or more data selections, further comprises:

prompting, by one or more computer processors, the quality evaluation of the one or more data selections, wherein the quality evaluation of the one or more data selections include at least one of ranking a data source credibility, ranking a data source prestige, ranking a data source relevance to a particular domain, and ranking a ratio of data updated based on data filtering; and

prompting, by one or more computer processors, to modify an existing data policy.

8. A computer program product for improving data collection and analytics utilizing off-line data auto-tagging and on-line fine granularity data tagging based on negotiations between a data provider and a data consumer, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to receive, by one or more computer processors, a data request from the data consumer;

program instructions to generate, by one or more computer processors, one or more attributes from the data request;

program instructions to search, by one or more computer processors, one or more data sources for data related to the one or more attributes;

program instructions to collect, by one or more computer processors, data related to the one or more attributes from the one or more data sources, wherein collecting the data related to the one or more attributes includes populating a data descriptor template for the data related to the one or more attributes with one or more descriptors;

program instructions to determine, by one or more computer processors, whether the collected data match one or more descriptors;

responsive to a determination that the collected data match the one or more descriptors, program instructions to create, by one or more computer processors, one or more annotated ontologies for the collected data by combining one or more ontologies defining a domain for the collected data with one or more modifiable data policy rules to describe the data and one or more handling requirements for the collected data based on the one or more modifiable data policy rules, wherein creating the one or more annotated ontologies includes annotating the one or more descriptors in the data descriptor template with at least one summary of one or more documents from the collected data, wherein annotating the one or more descriptors with the at least one summary includes associating summary information generated from searching the one or more data sources, applying data policy rules to the at least one summary, and modifying the at least one summary to conform to relevant data policy rules related to data cost, data topology and quantity of data;

program instructions to provide, by one or more computer processors, a hierarchical visualization of the one or more annotated ontologies for the collected data to the data consumer;

program instructions to receive, by one or more computer processors, one or more data selections from the one or more annotated ontologies; and

program instructions to update, by one or more computer processors, the one or more annotated ontologies for the collected data based, at least in part, on a quality evaluation of the one or more data selections.

9. The computer program product of claim 8, wherein program instructions to determine whether the collected data match one or more descriptors, further comprises at least one of:

program instructions to determine, by one or more computer processors, that the one or more attributes generated from the data request match the one or more descriptors associated with the collected data where at least one of one or more features of the collected data are related to at least one of the one or more attributes generated from the data request; and

program instructions to determine, by one or more computer processors, that the one or more attributes generated from the data request do not match the one or more descriptors associated with the collected data where at least one of one or more features of the collected data are not related to at least one of the one or more attributes generated from the data request.

10. The computer program product of claim 8, wherein program instructions to provide a hierarchical visualization of one or more ontologies for the collected data, further comprises:

program instructions to associate, by one or more computer processors, one or more user defined policy rules with one or more descriptors from the collected data, wherein associating the one or more user defined policy rules with one or more descriptors for the collected data includes determining a level of relevancy between policy rules and data descriptors.

11. The computer program product of claim 8, further comprises:

program instructions to filter, by one or more computer processors, data from the one or more ontologies for the collected data, at least in part, on the one or more data selections, a data consumer data policy, and a data provider data policy, wherein filtering includes cleaning data from the one or more ontologies by applying one or more modifiable data policy rules from the data consumer data policy and the data provider data policy.

12. The computer program product of claim 8, wherein program instructions to update the one or more ontologies for the collected data based, at least in part, on a quality evaluation of the one or more data selections, further comprises:

program instructions to prompt, by one or more computer processors, the quality evaluation of the one or more data selections, wherein the quality evaluation of the one or more data selections include at least one of ranking a data source credibility, ranking a data source prestige, ranking a data source relevance to a particular domain, and ranking a ratio of data updated based on data filtering; and

program instructions to prompt, by one or more computer processors, to modify an existing data policy.

13. A computer system for ontological policy based data collection, processing, and negotiation for data in view of analytics, the computer system comprising:

one or more hardware computer processors;

one or more computer readable storage media;

program instructions stored on at least one of the one or more computer readable storage media for execution by at least one of the one or more hardware computer processors, the program instructions comprising;

program instructions to receive, by one or more hardware computer processors, a data request from the data consumer;

program instructions to generate, by one or more hardware computer processors, one or more attributes from the data request;

program instructions to search, by one or more hardware computer processors, one or more data sources for data related to the one or more attributes;

program instructions to collect, by one or more hardware computer processors, data related to the one or more attributes from the one or more data sources, wherein collecting the data related to the one or more attributes includes populating a data descriptor template for the data related to the one or more attributes with one or more descriptors;

program instructions to determine, by one or more hardware computer processors, whether the collected data match one or more descriptors;

responsive to a determination that the collected data match the one or more descriptors, program instructions to create, by one or more hardware computer processors, one or more annotated ontologies for the collected data by combining one or more ontologies defining a domain for the collected data with one or more modifiable data policy rules to describe the data and one or more handling requirements for the collected data based on the one or more modifiable data policy rules, wherein creating the one or more annotated ontologies includes annotating the one or more descriptors in the data descriptor template with at least one summary of one or more documents from the collected data, wherein annotating the one or more descriptors with the at least one summary includes associating summary information generated from searching the one or more data sources, applying data policy rules to the at least one summary, and modifying the at least one summary to conform to relevant data policy rules related to data cost, data topology and quantity of data;

program instructions to provide, by one or more hardware computer processors, a hierarchical visualization of the one or more annotated ontologies for the collected data to the data consumer;

program instructions to receive, by one or more hardware computer processors, one or more data selections from the one or more annotated ontologies; and

program instructions to update, by one or more hardware computer processors, the one or more annotated ontologies for the collected data based, at least in part, on a quality evaluation of the one or more data selections.

14. The computer system of claim 13, wherein program instructions to provide a hierarchical visualization of one or more ontologies for the collected data, further comprises:

program instructions to associate, by one or more hardware computer processors, one or more user defined policy rules with one or more descriptors from the collected data, wherein associating the one or more user defined policy rules with one or more descriptors for the collected data includes determining a level of relevancy between policy rules and data descriptors.

15. The computer system of claim 13, further comprises:

program instructions to filter, by one or more hardware computer processors, data from the one or more ontologies for the collected data, at least in part, on the one or more data selections, a data consumer data policy, and a data provider data policy, wherein filtering includes cleaning data from the one or more ontologies by applying one or more modifiable data policy rules from the data consumer data policy and the data provider data policy.

16. The computer system of claim 13, wherein program instructions to update the one or more ontologies for the collected data based, at least in part, on a quality evaluation of the one or more data selections, further comprises:

program instructions to prompt, by one or more hardware computer processors, the quality evaluation of the one or more data selections, wherein the quality evaluation of the one or more data selections include at least one of ranking a data source credibility, ranking a data source prestige, ranking a data source relevance to a particular domain, and ranking a ratio of data updated based on data filtering; and

program instructions to prompt, by one or more hardware computer processors, to modify an existing data policy.