Managing data sets by reasoning over captured metadata

• US 8,838,556 B1
• Filed: 12/23/2011
• Issued: 09/16/2014
• Est. Priority Date: 08/30/2011
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Reiner, David Stephen, Hushon, John Daniel, Nanda, Nihar
• Primary Examiner(s): Ehichioya, Fred I
• Assistant Examiner(s): MENG, JAU SHYA
• Application Number:
  US13/336,613
• Time in Patent Office: 998 Days
• Field of Search: None
• US Class Current: 707/694
• CPC Class Codes: G06F 16/904 Browsing; Visualisation therefor for navigating the web G06F16/954; browsing optimisation for the web G06F16/957

Abstract

A data set manager is configured to interact with data processing elements of an information processing system. The data set manager comprises a metadata capture module configured to access or otherwise obtain metadata characterizing data sets associated with the data processing elements, a reasoning module configured to perform one or more reasoning operations on the metadata, and an action recommendation module configured to identify one or more recommended actions for the data processing elements based at least in part on results of the reasoning operations. The metadata characterizes properties of the data sets and relationships among the data sets, and may be defined in accordance with at least one of a specified ontology and a specified class. The data set manager and associated data processing elements may be implemented, by way of example, in cloud infrastructure of a cloud service provider, or on another type of processing platform.

Claims

1. An apparatus comprising:

at least one processing device comprising a memory and a processor coupled to the memory, the at least one processing device implementing a data set manager configured to interact with data processing elements of an information processing system;

the data set manager comprising;

a metadata capture module configured to obtain metadata characterizing data sets associated with said data processing elements;

a reasoning module configured to perform one or more reasoning operations on the metadata; and

an action recommendation module configured to identify one or more recommended actions for the data processing elements based at least in part on results of the one or more reasoning operations;

wherein the metadata characterizes properties of a given data set and relationships among the given data set and at least one other data set, the properties of the given data set comprising at least one property relating to a history of the data set and at least one property relating to a classification of the data set with respect to a given scheme;

wherein the metadata capture module comprises;

a data set discovery module coupled to one or more of the data processing elements and operative to obtain metadata characterizing the data sets therefrom; and

a knowledge query and assertion processor operative to process one or more asserted properties or relationships of the given data set in accordance with a model generated by a model manager, the model specifying structural, semantic and format characteristics of the metadata;

wherein the model manager is part of a set of common services of the data set manager and is operative;

to add or modify at least one of an ontology and a class utilized by the knowledge query and assertion processor; and

to add or modify a policy utilized by a policy engine of the action recommendation module;

wherein the knowledge query and assertion processor is operative to process a query or modify knowledge request directed to at least a portion of the metadata; and

wherein the action recommendation module comprises a policy engine configured to generate a processing directive to initiate the one or more recommended actions for one or more of the data processing elements based on results of the one or more reasoning operations and at least one policy selected from a policy repository.

2. The apparatus of claim 1 wherein at least portions of common services of the data set manager are utilized by the metadata capture module, the reasoning module and the action recommendation module.

3. The apparatus of claim 2 wherein the set of common services comprises at least one of a data set knowledge repository which stores at least a portion of the metadata, and the policy repository which stores a plurality of policies for use by a policy engine of the action recommendation module.

4. The apparatus of claim 1 wherein at least a portion of the metadata is defined at least in part in accordance with at least one of a specified ontology and a specified class.

5. The apparatus of claim 1 wherein the properties relating to a history of the given data set comprise information characterizing one or more of at least creation, ownership, location, derivation and transformation, and wherein the properties relating to the classification of the data set with respect to the given scheme comprise information characterizing one or more of content, state, importance, security, currency, volatility, confidentiality, certification and validity of that data set.

6. The apparatus of claim 1 wherein the relationships among the given data set and at least one other data set comprise information characterizing one or more of at least derivation, transformation, consistency, fidelity and substitutability of those data sets.

7. The apparatus of claim 1 wherein the one or more reasoning operations comprise one or more of:

determining if the given data set is suitable for a specified use;

determining a best one or more of the data sets for a specified use;

determining if a policy related to one or more of the data sets is satisfied;

determining if the given data set originates from a particular source;

determining if appropriate security safeguards have been applied to the given data set;

determining if another data set can be substituted for the given data set; and

determining if access controls of the given data set are appropriate to a current usage context.

8. The apparatus of claim 1 wherein the reasoning module comprises a knowledge manager and inference engine.

9. The apparatus of claim 1 wherein at least a subset of the data processing elements comprise one or more of security enforcement services, analytical applications, platform management services and infrastructure management services.

10. The apparatus of claim 1 wherein at least a subset of the data processing elements comprise processing devices of a processing platform of the information processing system.

11. A method comprising the steps of:

capturing metadata characterizing data sets associated with data processing elements of an information processing system;

performing one or more reasoning operations on the metadata; and

identifying one or more recommended actions for the data processing elements based at least in part on results of the one or more reasoning operations;

wherein the steps are performed by at least one processing device comprising a processor coupled to a memory;

wherein the metadata characterizes properties of a given data set and relationships among the given data set and at least one other data set, the properties of the given data set comprising at least one property relating to a history of the data set and at least one property relating to a classification of the data set with respect to a given scheme;

wherein the step of capturing metadata characterizing data sets comprises;

obtaining metadata characterizing the data sets from one or more of the data processing elements;

generating a model specifying structural, semantic and format characteristics of the metadata;

processing one or more asserted properties or relationships of the given data set in accordance with the model;

adding or modifying at least one of an ontology and a class relating to the data sets;

adding or modifying at least one policy relating to the data sets; and

processing a query or modify knowledge request directed to at least a portion of the metadata; and

wherein the step of identifying one or more recommended actions for the data processing elements comprises generating a processing directive to initiate the one or more recommended actions for one or more of the data processing elements based on results of the one or more reasoning operations and at least one policy selected from a policy repository.

12. The method of claim 11 wherein the step of performing one or more reasoning operations on the metadata comprises performing one or more of:

determining if the given data set is suitable for a specified use;

determining a best one or more of the data sets for a specified use;

determining if a policy related to one or more of the data sets is satisfied;

determining if the given data set originates from a particular source;

determining if appropriate security safeguards have been applied to the given data set;

determining if another data set can be substituted for the given data set; and

determining if access controls of the given data set are appropriate to a current usage context.

13. The method of claim 11 wherein the properties relating to a history of the given data set comprise information characterizing one or more of at least creation, ownership, location, derivation and transformation, and wherein the properties relating to the classification of the data set with respect to the given scheme comprise information characterizing one or more of content, state, importance, security, currency, volatility, confidentiality, certification and validity of that data set.

14. The method of claim 11 wherein the relationships among the given data set and at least one other data set comprise information characterizing one or more of at least derivation, transformation, consistency, fidelity and substitutability of those data sets.

15. The method of claim 11 wherein at least a subset of the data processing elements comprise one or more of security enforcement services, analytical applications, platform management services and infrastructure management services.

16. A computer program product comprising a non-transitory processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed cause a processing device to:

capture metadata characterizing data sets associated with data processing elements of an information processing system;

perform one or more reasoning operations on the metadata; and

identify one or more recommended actions for the data processing elements based at least in part on results of the one or more reasoning operations;

wherein the metadata characterizes properties of a given data set and relationships among the given data set and at least one other data set, the properties of the given data set comprising at least one property relating to a history of the data set and at least one property relating to a classification of the data set with respect to a given scheme;

wherein capturing metadata characterizing data sets comprises;

obtaining metadata characterizing the data sets from one or more of the data processing elements;

generating a model specifying structural, semantic and format characteristics of the metadata;

processing one or more asserted properties or relationships of the given data set in accordance with the model;

adding or modifying at least one of an ontology and a class relating to the data sets;

adding or modifying at least one policy relating to the data sets; and

processing a query or modify knowledge request directed to at least a portion of the metadata; and

wherein identifying one or more recommended actions for the data processing elements comprises generating a processing directive to initiate the one or more recommended actions for one or more of the data processing elements based on results of the one or more reasoning operations and at least one policy selected from a policy repository.

17. The computer program product of claim 16 wherein the properties relating to a history of the given data set comprise information characterizing one or more of at least creation, ownership, location, derivation and transformation, and wherein the properties relating to the classification of the data set with respect to the given scheme comprise information characterizing one or more of content, state, importance, security, currency, volatility, confidentiality, certification and validity of that data set.

18. The computer program product of claim 16 wherein the relationships among the given data set and at least one other data set comprise information characterizing one or more of at least derivation, transformation, consistency, fidelity and substitutability of those data sets.

19. The computer program product of claim 16 wherein at least a subset of the data processing elements comprise one or more of security enforcement services, analytical applications, platform management services and infrastructure management services.

20. The computer program product of claim 16 wherein the one or more reasoning operations comprise one or more of:

determining if the given data set is suitable for a specified use;

determining a best one or more of the data sets for a specified use;

determining if a policy related to one or more of the data sets is satisfied;

determining if the given data set originates from a particular source;

determining if appropriate security safeguards have been applied to the given data set;

determining if another data set can be substituted for the given data set; and

determining if access controls of the given data set are appropriate to a current usage context.