CONSOLIDATING ANALYTICS METADATA

• US 20140372378A1
• Filed: 03/11/2014
• Published: 12/18/2014
• Est. Priority Date: 06/13/2013
• Status: Active Grant

• Current Assignee: Entrust Corp.
• Original Assignee: DataGravity, Inc. (Entrust Corp.)
• Inventors: Long, Paula, McCall, Eric K., Bochkov, Dmitry, Zelikov, Misha, Panner, Bryan Keith
• Granted Patent: US 9,262,281 B2
• Time in Patent Office: Days
• US Class Current: 707/646
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1466 to make the backup process non-disruptive
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 11/2082 Data synchronisation
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A single system merges primary data storage, data protection, and intelligence. Intelligence is provided through in-line data analytics, and data intelligence and analytics are gathered on protected data and prior analytics, and stored in discovery points, all without impacting performance of primary storage. As data is written it is automatically mirrored for data protection as part of a High Availability (HA) process. Real-time analysis is done in-line with the HA processing, enabling a variety of data analytics. Data content can mined from within files or blocks. The gathered intelligence is used to tag objects with extended metadata, enabling both valuable search options and rapid restore options. Data recovery begins with metadata restoration, followed by near-instantaneous access to “hot” regions of data being restored, allowing site operation to continue or resume while a restore is ongoing.

Claims

1. A method for handling data in a data storage system comprising:

receiving a copy of data access requests and metadata concerning the data access requests;

executes further analytics on the metadata associated with the data access requests to provide further analytics data; and

consolidating the metadata and the further analytics for storage as consolidated metadata and further analytics data in a storage device.

2. The method of claim 1 wherein the copy of data access requests is received from a primary node, and an intelligence node provides the storage device for storing the consolidated metadata and further analytics data.

3. The method of claim 2 wherein the primary node and intelligence node are co-located.

4. The method of claim 2 wherein the primary node and intelligence node are located remotely from one another.

5. The method of claim 2 wherein a single processing node implements the primary node and intelligence node, and the intelligence node executes when the single processing node is not occupied with functions of the primary node.

6. The method of claim 1 wherein the metadata includes one or more of:

extended information concerning the data access request;

who initiated a data access request;

how many modifications have occurred;

aggregate size of the modifications;

a data object name;

data object owner;

access control list;

or

time of the data access request.

7. The method of claim 2 wherein the intelligence node further

directs the data access requests to an intelligence pool;

and

creates discovery points associating the consolidated metadata and further analytics data with an intelligence pool copy of the data.

8. The method of claim 7 wherein the intelligence node further

performs deeper analytics on the discovery points, including content extraction and/or

analysis of information stored in the discovery points.

9. The method of claim 7 wherein the intelligence node further

compares metadata related to two or more discovery points associated with data access requests occurring at two or more different times.

10. The method of claim 8 further comprising:

executing one or more rules on at least one of the change catalog, metadata, and extracted content information to perform an additional operation that include one or more of;

applying a rule that includes a filter to match content and an associated action;

tracking changes in the data or metadata over time;

applying sentiment attributes to an indexed document;

processing of a regular expression;

content transformation;

content analysis;

or

triggering an action based on detected access or content wherein the action further comprises one or more of;

data retention;

quarantine;

data extraction;

deletion;

data distribution;

alerts;

or

other actions.

11. The method of claim 10 wherein a result of executing one or more rules provides complex analytics stored as additional metadata appended to one or more change catalogs.

12. The method of claim 2 wherein the intelligence node further provides:

selective restore of the primary node using the analytics metadata associated with a discovery point.

13. The method of claim 12 wherein the selective restore is in response to discovering a predetermined pattern in the data or metadata.

14. The method of claim 2 wherein the intelligence node further controls replication of the data, associated analytics, and a linkage between the data and associated analytics to a remote location.

15. The method of claim 14 further comprising:

selective replication of analytics metadata at the remote location based on one or more of;

size of the analytics metadata;

complexity of a metadata production process;

priorities;

or

processing load.

16. A primary storage, high availability and data analytics system comprising:

one or more processors programmed to operate intelligence node software wherein the intelligence node software;

receives a copy of data access requests and extended metadata concerning the access requests;

directs the data access requests to physical storage media in an intelligence pool;

executes further analytics on the data associated with the data access request and extended metadata; and

consolidates the extended metadata and the further analytics for storage as consolidated metadata in the intelligence pool.

17. The system of claim 16 wherein the copy of data access requests is received from a primary node.

18. The system of claim 17 wherein the primary node and intelligence node software are co-located.

19. The system of claim 17 wherein the primary node and intelligence node software are located remotely from one another.

20. The system of claim 17 wherein a single processing node implements the primary node and intelligence node, and the intelligence node executes during when the node is not occupied with functions of the primary node.

21. The system of claim 16 wherein the extended metadata includes one or more of:

extended information concerning the data access request;

who initiated a data access request;

how many modifications have occurred;

aggregate size of the modifications;

a data object name;

data object owner;

access control list;

or

time of the data access request.

22. The system of claim 16 wherein the intelligence node software further

creates discovery points associating the change catalog and the further analytics with an intelligence pool copy of the data.

23. The system of claim 22 wherein the intelligence node software further

performs deeper analytics on the discovery points, including content extraction and/or analysis of the data.

24. The system of claim 22 wherein the intelligence node software further

compares metadata related to two or more discovery points associated with data access requests occurring at two or more different times.

25. The system of claim 16 wherein the intelligence node software further includes

a rule engine, for operating on at least one of the change catalog, extended metadata, and

content information to perform an additional operation that include one or more of:

applying a rule that includes a filter to match content and an associated action;

tracking changes in the data or metadata over time;

applying sentiment attributes to an indexed document;

processing of a regular expression;

content transformation;

content analysis;

or

triggering an action based on detected access or content wherein the action further comprises one or more of;

data retention;

quarantine;

data extraction;

deletion;

data distribution;

alerts;

or

other actions.

26. The system of claim 16 wherein the further analytics are provided by a rule engine and stored as additional metadata appended to one or more change catalogs.

27. The system of claim 16 wherein the intelligence node software further provides:

selective restore of the primary node using the analytics metadata associated with a discovery point.

28. The system of claim 27 wherein the selective restore is in response to discovering a predetermined pattern in the data or metadata.

29. The system of claim 16 wherein the intelligence node software further controls replication of the data, associated analytics, and a linkage between the data and associated analytics to a remote location.

30. The system of claim 29 wherein intelligence node software further comprises:

selective replication of analytics metadata at the remote location based on one or more of;

size of the analytics metadata;

complexity of a metadata production process;

priorities;

or

processing load.